Release version 9.3.0 (#789)

* Periodically rerun ordersync instead of running it only once on startup (#764) * Periodically rerun ordersync instead of running it only once on startup * Add a random jitter to ordersync delay * Change ordersyncApproxDelay to 1 hour * Add log message when starting ordersync service * Improve comment clarity * Update ganache addresses (#765) * Update Ganache DevUtils address * Update Ganache snapshot version * Update Dummy ERC1155 contract address * Fix typo * Add a polyfill for WebAssembly.instantateStreaming (#770) * Add a polyfill for WebAssembly.instantateStreaming * Fix linter error * Update changelog * Compare chain id env variable with rpc on startup (#733) * ADDS check that configured chain id matches RPC WIP * FIXES async error channel race condition and clean up code * FIXES remove unnecessary wait group, move chain id fetch into helper * FIXES use ParseBig256 for parsing chain id response from eth_chainId * Fix race condition in core.App.Start * Update changelog Co-authored-by: Alex Browne <stephenalexbrowne@gmail.com> * Add custom order filters documentation (#768) * Add custom order filters docs * Fix nits * Add doc to menu * Improve limitations description * Add example filters * Merge the two example sections * Fully Automate Document Generation (#771) * Automated the documentation update step * Ignored exported fields that shouldn't be documented * Addressed @albrow's review feedback * Revert "Addressed @albrow's review feedback" This reverts commit 75f48c0. * [expirationwatch] Fixes consistency bug (#773) * Fix small bug in expirationwatcher * Add a test to ensure that a "barely expired" order will be pruned * Update CHANGELOG with PR number * Fixes race condition in orderwatcher test (#774) * Improve scenario package (#779) * WIP Improve scenario package * Create orderopts package. Update scenario and ordervalidator to use it * Fix remaining ordervalidator tests * Update most orderwatcher tests * Support setting taker state and enable remaining orderwatcher tests * Fix remaining tests (only MultiAsset disabled) * Uncomment test that depends on on-chain state for a StaticCall order * Fix browser integration test * Remove unneeded ethClient parameter in a few places * Move sleep statement in order_watcher_test.go * Address Alex Towle's feedback * Remove old comment * Revalidate affected orders regardless of ERC721 approval operator (#782) * Increase sleep duration in TestPingPong (#785) * scenario: Add ability to create multiple orders in a single batch (#784) * scenario: Add ability to create multiple orders in a single batch * Fix some typos * Update changelog for PR #782 (#786) * p2p: Don't call HandleMessages if there are no messages to handle (#787) * Add missing regression test for ordersync pagination subprotocol (#788) * Add missing regression test for ordersync pagination subprotocol Also fixes a minor bug where the exponential backoff for ordersync was too long in some circumstances. * Remove some duplicated code in core_test.go * Set versions to 9.3.0 Co-authored-by: Fabio B <me@fabioberger.com> Co-authored-by: Kim Persson <kimpers@users.noreply.github.com> Co-authored-by: Alex Towle <jalextowle@gmail.com>
0xProject · Apr 20, 2020 · 555bbc1 · 555bbc1
1 parent 8afe758
commit 555bbc1
Show file tree

Hide file tree

Showing 45 changed files with 1,967 additions and 1,486 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -7,7 +7,7 @@ jobs:
       - image: circleci/golang:1.13.4-browsers
       - image: 0xorg/ganache-cli:istanbul
         environment:
-            VERSION: 5.1.0
+            VERSION: 6.2.4
             SNAPSHOT_NAME: 0x_ganache_snapshot
     resource_class: large
     steps:

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,20 @@
 
 This changelog is a work in progress and may contain notes for versions which have not actually been released. Check the [Releases](https://github.com/0xProject/0x-mesh/releases) page to see full release notes and more information about the latest released versions.
 
+
+## v9.3.0
+
+### Features ✅
+
+- Mesh now ensures on startup that the chain ID of your Ethereum RPC endpoint matches config.EthereumChainID [#733](https://github.com/0xProject/0x-mesh/pull/733).
+
+### Bug fixes 🐞
+
+- Fixed a compatibility issue in `@0x/mesh-browser-lite` for Safari and some other browsers [#770](https://github.com/0xProject/0x-mesh/pull/770).
+- Fixes an issue that would allow expired orders to be returned in `GetOrders` [773](http://github.com/0xProject/0x-mesh/pull/773)
+- Fixed a rare bug where ERC721 approval events could be missed [#782](https://github.com/0xProject/0x-mesh/pull/782)
+
+
 ## v9.2.1
 
 ### Bug fixes 🐞

diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-[![Version](https://img.shields.io/badge/version-9.2.1-orange.svg)](https://github.com/0xProject/0x-mesh/releases)
+[![Version](https://img.shields.io/badge/version-9.3.0-orange.svg)](https://github.com/0xProject/0x-mesh/releases)
 [![Docs](https://img.shields.io/badge/docs-website-yellow.svg)](https://0x-org.gitbook.io/mesh)
 [![Chat with us on Discord](https://img.shields.io/badge/chat-Discord-blueViolet.svg)](https://discord.gg/HF7fHwk)
 [![Circle CI](https://img.shields.io/circleci/project/0xProject/0x-mesh/master.svg)](https://circleci.com/gh/0xProject/0x-mesh/tree/master)

diff --git a/RELEASE_CHANGELOG.md b/RELEASE_CHANGELOG.md
@@ -1,11 +1,17 @@
 - [Docker image](https://hub.docker.com/r/0xorg/mesh/tags)
-- [README](https://github.com/0xProject/0x-mesh/blob/v9.2.1/README.md)
+- [README](https://github.com/0xProject/0x-mesh/blob/v9.3.0/README.md)
 
 ## Summary
 
+### Features ✅
+
+- Mesh now ensures on startup that the chain ID of your Ethereum RPC endpoint matches config.EthereumChainID [#733](https://github.com/0xProject/0x-mesh/pull/733).
+
 ### Bug fixes 🐞
 
-- Fixed a critical bug in the ordersync protocol which resulted in only 50% of existing orders being shared when a new peer joins the network. New orders are shared separately and were unaffected. [#760](https://github.com/0xProject/0x-mesh/pull/760).
+- Fixed a compatibility issue in `@0x/mesh-browser-lite` for Safari and some other browsers [#770](https://github.com/0xProject/0x-mesh/pull/770).
+- Fixes an issue that would allow expired orders to be returned in `GetOrders` [773](http://github.com/0xProject/0x-mesh/pull/773)
+- Fixed a rare bug where ERC721 approval events could be missed [#782](https://github.com/0xProject/0x-mesh/pull/782)
 
 
 
diff --git a/cmd/cut-release/main.go b/cmd/cut-release/main.go
@@ -5,13 +5,16 @@ import (
 	"fmt"
 	"io/ioutil"
 	"log"
+	"os"
 	"os/exec"
 	"regexp"
 	"strings"
 
 	"github.com/plaid/go-envvar/envvar"
 )
 
+var functionDocsTemplate = "\n# Functions\n\n## loadMeshStreamingForURLAsync\n▸ **loadMeshStreamingWithURLAsync**(`url`: `string`): *Promise‹`void`›*\n\n*Defined in [index.ts:7](https://github.com/0xProject/0x-mesh/blob/%s/packages/browser-lite/src/index.ts#L7)*\n\nLoads the Wasm module that is provided by fetching a url.\n\n**Parameters:**\n\nName | Type | Description |\n------ | ------ | ------ |\n`url` | `string` | The URL to query for the Wasm binary |\n\n<hr />\n\n## loadMeshStreamingAsync\n\n▸ **loadMeshStreamingAsync**(`response`: `Response | Promise<Response>`): *Promise‹`void`›*\n\n*Defined in [index.ts:15](https://github.com/0xProject/0x-mesh/blob/%s/packages/browser-lite/src/index.ts#L15)*\n\nLoads the Wasm module that is provided by a response.\n\n**Parameters:**\n\nName | Type | Description |\n------ | ------ | ------ |\n`response` | `Response &#124; Promise<Response>` | The Wasm response that supplies the Wasm binary |\n\n<hr />"
+
 type envVars struct {
 	// Version is the new release version to use
 	Version string `envvar:"VERSION"`
@@ -34,15 +37,7 @@ func main() {
 		log.Fatal(err)
 	}
 
-	// Generate documentation for the Typescript packages.
-	cmd = exec.Command("yarn", "docs:md")
-	cmd.Dir = "."
-	stdoutStderr, err = cmd.CombinedOutput()
-	if err != nil {
-		log.Print(string(stdoutStderr))
-		log.Fatal(err)
-	}
-
+	generateTypescriptDocs()
 	createReleaseChangelog(env.Version)
 }
 
@@ -67,6 +62,49 @@ func createReleaseChangelog(version string) {
 	}
 }
 
+func generateTypescriptDocs() {
+	// Generate the initial docs for the Typescript packages. These docs will
+	// be used to create the final set of docs.
+	cmd := exec.Command("yarn", "docs:md")
+	cmd.Dir = "."
+	stdoutStderr, err := cmd.CombinedOutput()
+	if err != nil {
+		log.Print(string(stdoutStderr))
+		log.Fatal(err)
+	}
+	commitHash, err := getDocsCommitHash("docs/browser-bindings/browser-lite/reference.md")
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	// Copy the browser-lite docs to the `@0x/mesh-browser` packages's `reference.md`
+	// file. These docs are the correct docs for the `@0x/mesh-browser` package.
+	cmd = exec.Command(
+		"cp",
+		"docs/browser-bindings/browser-lite/reference.md",
+		"docs/browser-bindings/browser/reference.md",
+	)
+	cmd.Dir = "."
+	stdoutStderr, err = cmd.CombinedOutput()
+	if err != nil {
+		log.Print(string(stdoutStderr))
+		log.Fatal(err)
+	}
+
+	// Create the documentation for the `loadMeshStreamingAsync` and the `loadMeshStreamingWithURLAsync`
+	// functions. Append these docs to the end of the existing browser-lite docs.
+	functionDocs := fmt.Sprintf(functionDocsTemplate, commitHash, commitHash)
+	f, err := os.OpenFile("docs/browser-bindings/browser-lite/reference.md",
+		os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
+	if err != nil {
+		log.Fatal(err)
+	}
+	defer f.Close()
+	if _, err := f.WriteString(functionDocs); err != nil {
+		log.Fatal(err)
+	}
+}
+
 // Update the version string in all files that must be updated for a new release
 func updateHardCodedVersions(version string) {
 	// Update `packages/rpc-client/package.json`
@@ -134,6 +172,22 @@ func updateFileWithRegex(filePath string, regex string, replacement string) {
 	}
 }
 
+func getDocsCommitHash(docsPath string) (string, error) {
+	dat, err := ioutil.ReadFile(docsPath)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	regex := "https://github.com/0xProject/0x-mesh/blob/([a-f0-9]+)/"
+	var re = regexp.MustCompile(regex)
+	matches := re.FindStringSubmatch(string(dat))
+
+	if len(matches) < 2 {
+		return "", errors.New("No contents found")
+	}
+	return matches[1], nil
+}
+
 func getFileContentsWithRegex(filePath string, regex string) (string, error) {
 	dat, err := ioutil.ReadFile(filePath)
 	if err != nil {

diff --git a/constants/constants.go b/constants/constants.go
@@ -58,7 +58,7 @@ var GanacheAccountToPrivateKey = map[common.Address][]byte{
 var GanacheDummyERC721TokenAddress = common.HexToAddress("0x07f96aa816c1f244cbc6ef114bb2b023ba54a2eb")
 
 // GanacheDummyERC1155MintableAddress is the dummy ERC1155 token address in the Ganache snapshot
-var GanacheDummyERC1155MintableAddress = common.HexToAddress("0x8d42e38980ce74736c21c059b2240df09958d3c8")
+var GanacheDummyERC1155MintableAddress = common.HexToAddress("0x038f9b392fb9a9676dbaddf78ea5fdbf6c7d9710")
 
 // ErrInternal is used whenever we don't wish to expose internal errors to a client
 var ErrInternal = errors.New("internal error")

diff --git a/core/core.go b/core/core.go
@@ -60,13 +60,28 @@ const (
 	estimatedNonPollingEthereumRPCRequestsPer24Hrs = 50000
 	// logStatsInterval is how often to log stats for this node.
 	logStatsInterval = 5 * time.Minute
-	version          = "9.2.1"
+	version          = "9.3.0"
 	// ordersyncMinPeers is the minimum amount of peers to receive orders from
 	// before considering the ordersync process finished.
-	ordersyncMinPeers            = 5
-	paginationSubprotocolPerPage = 500
+	ordersyncMinPeers = 5
+	// ordersyncApproxDelay is the approximate amount of time to wait between each
+	// run of the ordersync protocol (as a requester). We always request orders
+	// immediately on startup. This delay only applies to subsequent runs.
+	ordersyncApproxDelay = 1 * time.Hour
 )
 
+// privateConfig contains some configuration options that can only be changed from
+// within the core package. Intended for testing purposes.
+type privateConfig struct {
+	paginationSubprotocolPerPage int
+}
+
+func defaultPrivateConfig() privateConfig {
+	return privateConfig{
+		paginationSubprotocolPerPage: 500,
+	}
+}
+
 // Note(albrow): The Config type is currently copied to browser/ts/index.ts. We
 // need to keep both definitions in sync, so if you change one you must also
 // change the other.
@@ -184,6 +199,7 @@ type snapshotInfo struct {
 
 type App struct {
 	config                    Config
+	privateConfig             privateConfig
 	peerID                    peer.ID
 	privKey                   p2pcrypto.PrivKey
 	node                      *p2p.Node
@@ -209,6 +225,10 @@ type App struct {
 var setupLoggerOnce = &sync.Once{}
 
 func New(config Config) (*App, error) {
+	return newWithPrivateConfig(config, defaultPrivateConfig())
+}
+
+func newWithPrivateConfig(config Config, pConfig privateConfig) (*App, error) {
 	// Configure logger
 	// TODO(albrow): Don't use global variables for log settings.
 	setupLoggerOnce.Do(func() {
@@ -389,6 +409,7 @@ func New(config Config) (*App, error) {
 	app := &App{
 		started:                   make(chan struct{}),
 		config:                    config,
+		privateConfig:             pConfig,
 		privKey:                   privKey,
 		peerID:                    peerID,
 		chainID:                   config.EthereumChainID,
@@ -576,6 +597,27 @@ func (app *App) Start(ctx context.Context) error {
 		orderWatcherErrChan <- app.orderWatcher.Watch(innerCtx)
 	}()
 
+	// Ensure that RPC client is on the same ChainID as is configured with ETHEREUM_CHAIN_ID
+	chainIDMismatchErrChan := make(chan error, 1)
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		defer func() {
+			log.Debug("closing chainID checker")
+		}()
+
+		chainID, err := app.getEthRPCChainID(innerCtx)
+		if err != nil {
+			chainIDMismatchErrChan <- err
+			return
+		}
+
+		configChainID := app.config.EthereumChainID
+		if int64(configChainID) != chainID.Int64() {
+			chainIDMismatchErrChan <- fmt.Errorf("ChainID mismatch between RPC client (chainID: %d) and configured environment variable ETHEREUM_CHAIN_ID: %d", chainID, configChainID)
+		}
+	}()
+
 	// Note: this is a blocking call so we won't continue set up until its finished.
 	blocksElapsed, err := app.blockWatcher.FastSyncToLatestBlock(innerCtx)
 	if err != nil {
@@ -648,7 +690,7 @@ func (app *App) Start(ctx context.Context) error {
 
 	// Register and start ordersync service.
 	ordersyncSubprotocols := []ordersync.Subprotocol{
-		NewFilteredPaginationSubprotocol(app, paginationSubprotocolPerPage),
+		NewFilteredPaginationSubprotocol(app, app.privateConfig.paginationSubprotocolPerPage),
 	}
 	app.ordersyncService = ordersync.New(innerCtx, app.node, ordersyncSubprotocols)
 	orderSyncErrChan := make(chan error, 1)
@@ -658,7 +700,13 @@ func (app *App) Start(ctx context.Context) error {
 		defer func() {
 			log.Debug("closing ordersync service")
 		}()
-		if err := app.ordersyncService.GetOrders(innerCtx, ordersyncMinPeers); err != nil {
+		log.WithFields(map[string]interface{}{
+			"approxDelay":  ordersyncApproxDelay,
+			"perPage":      app.privateConfig.paginationSubprotocolPerPage,
+			"subprotocols": []string{"FilteredPaginationSubProtocol"},
+		}).Info("starting ordersync service")
+
+		if err := app.ordersyncService.PeriodicallyGetOrders(innerCtx, ordersyncMinPeers, ordersyncApproxDelay); err != nil {
 			orderSyncErrChan <- err
 		}
 	}()
@@ -703,47 +751,60 @@ func (app *App) Start(ctx context.Context) error {
 	log.Info("core.App was started")
 	close(app.started)
 
+	// Wait for all other goroutines to close.
+	appClosed := make(chan struct{})
+	go func() {
+		wg.Wait()
+		close(appClosed)
+	}()
+
 	// If any error channel returns a non-nil error, we cancel the inner context
 	// and return the error. Note that this means we only return the first error
 	// that occurs.
-	select {
-	case err := <-p2pErrChan:
-		if err != nil {
-			log.WithError(err).Error("p2p node exited with error")
-			cancel()
-			return err
-		}
-	case err := <-orderWatcherErrChan:
-		if err != nil {
-			log.WithError(err).Error("order watcher exited with error")
-			cancel()
-			return err
-		}
-	case err := <-blockWatcherErrChan:
-		if err != nil {
-			log.WithError(err).Error("block watcher exited with error")
-			cancel()
-			return err
-		}
-	case err := <-ethRPCRateLimiterErrChan:
-		if err != nil {
-			log.WithError(err).Error("ETH JSON-RPC ratelimiter exited with error")
-			cancel()
-			return err
-		}
-	case err := <-orderSyncErrChan:
-		if err != nil {
-			log.WithError(err).Error("ordersync service exited with error")
-			cancel()
-			return err
+	for {
+		select {
+		case err := <-p2pErrChan:
+			if err != nil {
+				log.WithError(err).Error("p2p node exited with error")
+				cancel()
+				return err
+			}
+		case err := <-orderWatcherErrChan:
+			if err != nil {
+				log.WithError(err).Error("order watcher exited with error")
+				cancel()
+				return err
+			}
+		case err := <-blockWatcherErrChan:
+			if err != nil {
+				log.WithError(err).Error("block watcher exited with error")
+				cancel()
+				return err
+			}
+		case err := <-ethRPCRateLimiterErrChan:
+			if err != nil {
+				log.WithError(err).Error("ETH JSON-RPC ratelimiter exited with error")
+				cancel()
+				return err
+			}
+		case err := <-orderSyncErrChan:
+			if err != nil {
+				log.WithError(err).Error("ordersync service exited with error")
+				cancel()
+				return err
+			}
+		case err := <-chainIDMismatchErrChan:
+			if err != nil {
+				log.WithError(err).Error("ETH chain id matcher exited with error")
+				cancel()
+				return err
+			}
+		case <-appClosed:
+			// If we reached here it means we are done and there are no errors.
+			log.Debug("app successfully closed")
+			return nil
 		}
 	}
-
-	// Wait for all goroutines to exit. If we reached here it means we are done
-	// and there are no errors.
-	wg.Wait()
-	log.Debug("app successfully closed")
-	return nil
 }
 
 func (app *App) periodicallyCheckForNewAddrs(ctx context.Context, startingAddrs []ma.Multiaddr) {