Ginkgo tests parallel #712

nytzuga · 2023-06-30T21:06:16Z

Why this should be merged

This PR runs all the Ginkgo tests in parallel, making the whole testing time twice as fast.

How this works

Ginkgo support running tests in parallel. This PR introduces a synchronized way to start an Avalanche Go node once, and setup one subnet for each tests to be run, and share those subnets IDs with the other processes.

How this was tested

We should have a green pipeline.

How is this documented

darioush

Could we either update the PR title to not include wip if it's ready or mark as draft if it is wip?
Also would you please update the PR description?

tests/precompile/solidity/suites.go

tests/utils/subnet.go

aaronbuchwald · 2023-07-13T00:50:09Z

How this was tested

We should have a green pipeline.

Nice

aaronbuchwald · 2023-07-13T00:53:52Z

tests/precompile/solidity/suites.go

+	subnetsBytes, err := json.Marshal(subnets)
+	gomega.Expect(err).NotTo(gomega.HaveOccurred())
+	compileContracts()
+	return subnetsBytes
+}, func(data []byte) {
+	err := json.Unmarshal(data, &subnetsIds)
+	gomega.Expect(err).NotTo(gomega.HaveOccurred())
+})


This is really nice. Putting this right here, this code is really clear that the SynchronizedBeforeSuite function generates this result and then the next function running in every process will unmarshal that result to ensure that this variable is set correctly in every process ginkgo is running.

aaronbuchwald · 2023-07-13T01:04:25Z

tests/utils/command.go

@@ -57,35 +56,3 @@ func RegisterPingTest() {
 		gomega.Expect(healthy.Healthy).Should(gomega.BeTrue())
 	})
 }
-
-// RegisterNodeRun registers a before suite that starts an AvalancheGo process to use for the e2e tests


We started using this RegisterNodeRun function here: https://github.com/ava-labs/precompile-evm/blob/main/tests/precompile/precompile_test.go#L18 to support builders that want to develop their own precompiles without maintaining a full fork of Subnet-EVM.

Could we add back this functionality, so that this PR doesn't break Precompile-EVM and then we can make a separate GitHub issue to leverage the same change to parallelize precompile e2e tests inside of Precompile-EVM as well?

I don't think we need to block this PR to figure out the best way to do that, so we should just make sure this doesn't break Precompile-EVM in the meantime.

cc @ceyonur

Great feedback and context. I redo the PR without touching this function and the final PR is much smaller.

yea I agree. Optimistically, changes here should not affect current non-parallel tests and they should be able to run fine with changes here. I.e they should be running fine without ginkgo run -procs=5. If it's not possible, then we should have different functions for parallel testing, still compatible with sub-functions like running hardhat tests etc.

aaronbuchwald · 2023-07-13T01:06:29Z

tests/precompile/solidity/suites.go

+// Timeout to boot the AvalancheGo node
+var bootAvalancheNodeTimeout = 5 * time.Minute
+
+// Timeout for the health API to check the AvalancheGo is ready
+var healthCheckTimeout = 5 * time.Second
+
+// At boot time subnets are created, one for each test suite. This global
+// variable has all the subnets IDs that can be used.
+//
+// One process creates the AvalancheGo node and all the subnets, and these
+// subnets IDs are passed to all other processes and stored in this global
+// variable
+var subnetsIds map[string]string
+
+// Keep track of the AvalancheGo external bash script, it is null for most
+// processes except the first process that starts AvalancheGo
+var startCmd *cmd.Cmd


Could we put this inside one var block and update subnetsIds to subnetIDs ?

Unfortunately, we don't have a style guide for this stuff 😅

happy to make a follow on PR if you prefer

tests/precompile/solidity/suites.go

aaronbuchwald

Left a couple of optional nits and one requested change to add back the exported function RegisterNodeRun

Great work!

ceyonur

I'm really hyped up for those parallel tests 🚀 We should just make sure that we're not breaking our existing tests as Subnet-EVM can be used as a library for e2e tests as well.

ceyonur · 2023-07-14T09:09:26Z

tests/utils/command.go

+		for _, file := range files {
+			basename := filepath.Base(file)
+			index := basename[:len(basename)-5]
+			subnets[index] = CreateNewSubnet(ctx, file)


can we create those subnets in parallel? i.e in their old places?

No, because they share the same wallet and if we do that we will have race conditions.

ceyonur · 2023-07-14T09:11:08Z

tests/utils/command.go

+
+		subnetsBytes, err := json.Marshal(subnets)
+		gomega.Expect(err).NotTo(gomega.HaveOccurred())
+		compileContracts()


this is too specific for this function. We should not assume subtests will use a contract test.

I'll move this compiling functions elsewhere.

ceyonur · 2023-07-14T09:17:29Z

scripts/run_ginkgo.sh


 # By default, it runs all e2e test cases!
 # Use "--ginkgo.skip" to skip tests.
 # Use "--ginkgo.focus" to select tests.
-./tests/precompile/precompile.test \
+TEST_SOURCE_ROOT="$TEST_SOURCE_ROOT" ginkgo run -procs=5 tests/precompile \


Why do we need TEST_SOURCE_ROOT?

Because the cwd is different if we compile and run the tests from a binary vs invoking them from the cli.

Maybe add a comment indicating why -procs=5 is preferable to letting ginkgo decide based on the number of cpus available?

ceyonur · 2023-07-14T09:20:02Z

tests/utils/command.go

@@ -57,35 +56,3 @@ func RegisterPingTest() {
 		gomega.Expect(healthy.Healthy).Should(gomega.BeTrue())
 	})
 }
-
-// RegisterNodeRun registers a before suite that starts an AvalancheGo process to use for the e2e tests


yea I agree. Optimistically, changes here should not affect current non-parallel tests and they should be able to run fine with changes here. I.e they should be running fine without ginkgo run -procs=5. If it's not possible, then we should have different functions for parallel testing, still compatible with sub-functions like running hardhat tests etc.

.github/workflows/ci.yml

nytzuga · 2023-07-14T18:19:32Z

.github/workflows/ci.yml

@@ -68,7 +68,7 @@ jobs:
      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
-          node-version: "18.x"
+          node-version: "18.15"


I had to downgrade due to NomicFoundation/hardhat#3877

nytzuga · 2023-07-14T18:19:40Z

contracts/package.json

@@ -41,6 +41,6 @@
  },
  "engines": {
    "npm": ">6.0.0",
-    "node": ">=18.16.0"
+    "node": ">=18.15.0"


I had to downgrade due to NomicFoundation/hardhat#3877

aaronbuchwald · 2023-07-14T19:10:26Z

tests/precompile/solidity/suites.go

@@ -22,35 +22,35 @@ var _ = ginkgo.Describe("[Precompiles]", ginkgo.Ordered, func() {
 		ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
 		defer cancel()

-		utils.RunDefaultHardhatTests(ctx, "contract_native_minter")
+		utils.RunDefaultHardhatTests(ctx, utils.SubnetsIDs["contract_native_minter"], "contract_native_minter")


The variable that needs to be passed in here is the blockchainID as opposed to the subnetID, could we change the name of this variable?

aaronbuchwald · 2023-07-14T19:11:18Z

tests/utils/subnet.go

-	genesisFilePath := fmt.Sprintf("./tests/precompile/genesis/%s.json", test)
-
-	blockchainID := CreateNewSubnet(ctx, genesisFilePath)
+func RunDefaultHardhatTests(ctx context.Context, blockchainID string, test string) {


could we update the comment here to remove the assumptions that no longer apply?

Use SynchronizedBeforeSuite to create the node and all the subnet that are needed to run tests. Pass the blockchainID to each work processor and run the tests in parallel to reduce runtime

aaronbuchwald

LGTM

darioush · 2023-07-14T22:30:05Z

contracts/package.json

@@ -28,7 +28,7 @@
  },
  "license": "BSD-3-Clause",
  "scripts": {
-    "build": "rm -rf dist/ && tsc -b",
+    "build": "rm -rf dist/ && tsc -b && npx hardhat compile --network local",


curious why this is needed?

I was debugging an issue where the hardhat compile would just give up with no error, so I moved the compilation earlier. I thought it was because we would using the compiler in parallel.

This was the issue though NomicFoundation/hardhat#3877

darioush · 2023-07-14T22:30:47Z

tests/utils/command.go

+var BlockchainIDs map[string]string
+
+// Timeout to boot the AvalancheGo node
+var bootAvalancheNodeTimeout = 5 * time.Minute
+
+// Timeout for the health API to check the AvalancheGo is ready
+var healthCheckTimeout = 5 * time.Second


style nit: we should define top-level vars in a block where possible, also seems the latter 2 could be constants

darioush · 2023-07-14T22:31:21Z

tests/utils/command.go

-	_ = ginkgo.BeforeSuite(func() {
-		ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
+
+	// Our test suite runs in a separated processes, ginkgo hasI


Suggested change

// Our test suite runs in a separated processes, ginkgo hasI

// Our test suite runs in separate processes, ginkgo has

darioush · 2023-07-14T22:32:10Z

tests/utils/command.go

-		ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
+
+	// Our test suite runs in a separated processes, ginkgo hasI
+	// SynchronizedBeforeSuite() which is promised to run once, and its output is


Suggested change

// SynchronizedBeforeSuite() which is promised to run once, and its output is

// SynchronizedBeforeSuite() which is run once, and its return value is

darioush · 2023-07-14T22:32:30Z

tests/utils/command.go

+	// SynchronizedBeforeSuite() which is promised to run once, and its output is
+	// passed over to each worker.
+	//
+	// In here an AvalancheGo node instance is started, and subnets are created for


Suggested change

// In here an AvalancheGo node instance is started, and subnets are created for

// Here an AvalancheGo node is started, and subnets are created for

darioush · 2023-07-14T22:34:19Z

tests/utils/subnet.go

+	log.Info("Executing HardHat tests on a new blockchain", "blockchainID", blockchainID, "test", test)
+	log.Info("Using subnet", "ChainURI", chainURI)


could we merge these 2 logs?

Improve tests based on feedback from #712

* Rename test_ to step_ for clarity (ava-labs#636) * Codeowners fix (specific dir overrides global) (ava-labs#639) * rename state upgrade test (ava-labs#641) * plugin/evm: fix godoc typo (ava-labs#647) Signed-off-by: Gyuho Lee <gyuho.lee@avalabs.org> * plugin/evm: remove unncessary tx.Hash op in map iter (ava-labs#648) Signed-off-by: Gyuho Lee <gyuho.lee@avalabs.org> * deployExampleRewardManager.ts hardhat script fixed (ava-labs#650) * Migrate HardHat tests to DS-tests (Solidity tests) (ava-labs#601) * Increase trie dirty default cache size to 512MB (ava-labs#656) * export gingkgo suites (ava-labs#659) * export gingkgo suites * remove precompiles description from ping tests * update package.json * fix url in package json * change package name * Publish npm module for contract-examples (ava-labs#663) * export gingkgo suites * remove precompiles description from ping tests * update package.json * fix url in package json * change package name * prepare for module publishing * use npm ci * preload ethers * nits * Fix broken path to docs in README. (ava-labs#665) * Fix contract examples package json (ava-labs#672) * rename contract-examples to contracts & fix pack issue * allow custom CMD to be run on blockchains for e2e tests * move default hardhat test to utils * fix comment * add more comments * change folder reference * Bump got and ethereum-waffle in /contracts (ava-labs#680) Removes [got](https://github.com/sindresorhus/got). It's no longer used after updating ancestor dependency [ethereum-waffle](https://github.com/EthWorks/Waffle). These dependencies need to be updated together. Removes `got` Updates `ethereum-waffle` from 3.4.4 to 4.0.10 - [Release notes](https://github.com/EthWorks/Waffle/releases) - [Commits](https://github.com/EthWorks/Waffle/compare/ethereum-waffle@3.4.4...ethereum-waffle@4.0.10) --- updated-dependencies: - dependency-name: got dependency-type: indirect - dependency-name: ethereum-waffle dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump flat and hardhat in /contracts (ava-labs#679) Bumps [flat](https://github.com/hughsk/flat) to 5.0.2 and updates ancestor dependency [hardhat](https://github.com/nomiclabs/hardhat). These dependencies need to be updated together. Updates `flat` from 4.1.1 to 5.0.2 - [Commits](hughsk/flat@4.1.1...5.0.2) Updates `hardhat` from 2.6.1 to 2.15.0 - [Release notes](https://github.com/nomiclabs/hardhat/releases) - [Commits](https://github.com/nomiclabs/hardhat/compare/hardhat-core-v2.6.1...hardhat@2.15.0) --- updated-dependencies: - dependency-name: flat dependency-type: indirect - dependency-name: hardhat dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix core benchmarks (ava-labs#697) * remove unused line in test (ava-labs#695) * bump version to v0.5.2 (ava-labs#681) * bump version to v0.5.2 * downgrade avalanchego to 1.10.2 * update compatibility * Remove unused ethereum-waffle and ethereumjs-tx pkgs (ava-labs#685) * move state upgrade related functions to own file (ava-labs#684) * Update load test to use ANR for network setup (ava-labs#579) * Update load test to use ANR for network setup * Remove ToWebsocketURI that relies on strings feature added in go1.20 * Update load test to use ENV var from CI if populated * Move env var logic into NewDefaultANRConfig * Add init functions and clean up comments * Remove unnecessary helper * Remove unneeded cmd * Cleanup * Address PR comments * Move 2 min timeout for SetupNetwork to the start of the function * Turn list of txs into tx sequence (ava-labs#683) * Turn list of txs into tx sequence * Add agent comments * Remove unused code and address comments * add some more comments * address more comments' * lint * Remove :::warning --------- Co-authored-by: Anusha <anushachillara@gmail.com> * Minor simulator fixes (ava-labs#703) * add benchmark to template (ava-labs#705) * add benchmark to template * fix comment * Add metrics for gas price / usage (ava-labs#700) Co-authored-by: aaronbuchwald <aaron.buchwald56@gmail.com> * added config setting to prune warpdb (ava-labs#702) * added config setting to prune warpdb * refactored warp prune uint test * message * fixed up nits * fixed nits * fixed nits * Update plugin/evm/config.go * Add wrapped validator state for primary to subnet warp messages (ava-labs#708) * Add wrapped validator state for primary to subnet warp message verification * Address PR comment * upload network runner artifact after e2e test run (ava-labs#709) * Differentiate mandatory/optional Subnet-EVM Network Upgrades (ava-labs#640) * remove network upgrades from upgrade configs and enforce them in vm init * fix tests * remove unused var * remove extra shutdown * copy test chain config when modify * use SimulatedTestChainConfig for simulated tests * copy modified test chain config * add optional upgrades * Add tests * fix comments * use unmarshal in test * unexport functions * add commented out test * add TODO comment * add blockchain.Stop as defer to test (ava-labs#714) Co-authered-by: najeal <haim.nathan@icloud.com> Co-authored-by: Ceyhun Onur <ceyhun.onur@avalabs.org> * Add support for predicate gas into IntrinsicGas (ava-labs#713) * Add support for predicate gas * Address review * Fix test and add test for PredicateGas * Address comments * Add signature aggregation helpers (ava-labs#711) * Add signature aggregation helpers * Fix * Move WarpQuorumDenominator into new file * Address comments * fix var rename * Improve comments * Address comments * Add predicate packing helper (ava-labs#710) * Add predicate packing helper * move predicate code and add readme * fix moved import * fix merge * Run ginkgo precompile tests in parallel (ava-labs#712) Use SynchronizedBeforeSuite to create the node and all the subnet that are needed to run tests. Pass the blockchainID to each work processor and run the tests in parallel to reduce runtime * Fix BlockContext comment (ava-labs#731) * precompile accepter should take logIdx (ava-labs#728) * precompile accepter should take logIdx * minimize mock * copyright yr * add comments * Improve tests (ava-labs#733) Improve tests based on feedback from ava-labs#712 * Fix precompile test gen template (ava-labs#737) * fix Benchmark * fix spelling/references in Benchmark * Clean up gossip (ava-labs#744) * Remove call from precompile (ava-labs#748) * Fix nil types tmpl (ava-labs#751) * add more nil types and fix int types * readd hash type * remove convertToNil * Sync coreth v0.12.4 rc0 (ava-labs#694) * apply diff to rc0 * replace coreth with subnet-evm * go mod tidy * remove must commit * remove apricots * replace coreth with subnet-evm * rename banff and cortina * test memory db * add dUpgrade * fix errors * sync with v0.12.4-rc0 * more diffs * change templates * readd preparePredicateSlots * run goimports * readd tx allow list check to txpool * Fix tests * readd subnet evm check for tests * fix tests * add gas limit & coinbase fakers * Fix timestamp types * check error from newblockchain * fix gas limit in simulated genesis * downgrade ci to go 1.19 * gofmt files * rebump go to 1.20 * bump anr to latest * bump anr to latest * prepare predicate storage slots in Subnet-EVM * uncomment tests * change order of airdrop memory cleanup * add utils import * Coreth 0.12.4 x apply (ava-labs#720) Co-authored-by: Darioush Jalali <darioush.jalali@avalabs.org> * move instruction sets * revert unpack changes * reduce diffs * move genesis verify to vm (ava-labs#753) * move genesis verify to vm * fix comment * set a correct default from genesis * fix log * revert change to vm.go --------- Co-authored-by: Darioush Jalali <darioush.jalali@avalabs.org> * avoid adding/removing GasLimits in tests (ava-labs#755) * move genesis verify to vm * fix comment * set a correct default from genesis * fix log * revert change to vm.go * avoid adding/removing GasLimits in tests * nit --------- Co-authored-by: Ceyhun Onur <ceyhun.onur@avalabs.org> --------- Co-authored-by: Darioush Jalali <darioush.jalali@avalabs.org> * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release (ava-labs#757) * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release * bump anr version to v1.7.1 * Migrate ABI unpack fix (ava-labs#750) * change sprintf verb and use values (ava-labs#760) * Generalize precompile test suite (ava-labs#759) * restore old functions, generalize paths in sync subnet creation * add subnet suit struct and methods * move constants * fix reviews * add comment * Update subnet.go Signed-off-by: Ceyhun Onur <ceyhun.onur@avalabs.org> --------- Signed-off-by: Ceyhun Onur <ceyhun.onur@avalabs.org> * Add warp contract implementation (ava-labs#718) * Add warp contract implementation * Cleanup predicate test * Fix new function signature * Replace invalid fuzz test with unit test * Add chain config to enable warp API for warp e2e test * remove unused var * Add experimental warning and move warp precompile to x/ package * fix warning label * Fix warning * vm test nits * Improve sendWarpMessenger sol comment * more vm warp test nits * Move warp params into params package * More vm warp test nits * Address more PR comments * Remove triggerTx2 * Add check for expected topics from sendWarpMessage log * Fix config test * Fix incorrect replace * remove unnecessary echo * Address comments * Address comments * Address PR comments * Improve comments * Convert [32]byte type to common.Hash * Add base cost for getVerifiedWarpMessage * fix require equal type check * Fix updated awm message format * Update warp message format * Move IWarpMessenger.sol to interfaces/ * Add newline to warp genesis * uncomment assertion * Fix broken links in README * Add metrics to simulator (ava-labs#706) * add tps * get block build metrics * Add issuance time, confirmed time, issued-> acceptance time, verify time * Log times at end of batch * cleaner * address comments * remove unused code block * avoids taking len of channel * nits * pass in stringID * move to loader * remove unused field * revert file back * cleaner * lint * make it work for ws or for rpc * protect * endpoint * no return on defer * sep to a funciton * have blockchainidstr passed in * typo * pass in metrics through config * address comments * address more comments and edit err policy of metrics functions * add more logging to load_test * typo * better check * fix endpoints * typo: * individual * histogram * address feedback: * remove metrics from default * address comments * simplify time metrics * better explanation * address comments * address comments * cleanup * more cleanup * rename vars for clarity * ws * cleanup * address comments * ws * expose metrics add flag * fix blocking issue of http server and gracefully stop it * cleanup * use constant * add issuance to confirmation metrics * ws * simplify metrics server * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release (ava-labs#757) * Bump avalanchego to v1.10.5 and bump Subnet-EVM for v0.5.3 release * bump anr version to v1.7.1 * handle control c * print out output * clean up * clean up * remove go routines to close client * address comments * memory leak * fix * print * e2e test: connect to appropriate chain (ava-labs#771) * Bump avalanchego dep and update version for next release (ava-labs#770) * Bump avalanchego dep and update version for next release * Update cache sizes * Update flag in run script to replace staking-enabled=false * Update block builder to enforce predicate one tx at a time (ava-labs#773) * Update block builder to enforce predicate one tx at a time * remove dead code * Update miner/worker.go Co-authored-by: Darioush Jalali <darioush.jalali@avalabs.org> Signed-off-by: aaronbuchwald <aaron.buchwald56@gmail.com> --------- Signed-off-by: aaronbuchwald <aaron.buchwald56@gmail.com> Co-authored-by: Darioush Jalali <darioush.jalali@avalabs.org> * update iocorders address * remove state-sync-enabled from chain config * 1s block ticker * flush hubbleFeed when len > 512 * fix compilation err --------- Signed-off-by: Gyuho Lee <gyuho.lee@avalabs.org> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Ceyhun Onur <ceyhun.onur@avalabs.org> Signed-off-by: aaronbuchwald <aaron.buchwald56@gmail.com> Co-authored-by: Richard Pringle <rpring9@gmail.com> Co-authored-by: Darioush Jalali <darioush.jalali@avalabs.org> Co-authored-by: Ceyhun Onur <ceyhun.onur@avalabs.org> Co-authored-by: Gyuho Lee <gyuho.lee@avalabs.org> Co-authored-by: İzzet Emre Demir <izzetemredemir@gmail.com> Co-authored-by: aaronbuchwald <aaron.buchwald56@gmail.com> Co-authored-by: Gabriel Cardona <gabriel@avalabs.org> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anusha <anushachillara@gmail.com> Co-authored-by: Cesar <137245636+nytzuga@users.noreply.github.com> Co-authored-by: morrisettjohn <60852062+morrisettjohn@users.noreply.github.com> Co-authored-by: nathan haim <nathan.haim@free.fr> Co-authored-by: Martin Eckardt <m.eckardt@outlook.com> Co-authored-by: rodrigo <77309055+RodrigoVillar@users.noreply.github.com> Co-authored-by: Anusha <63559942+anusha-ctrl@users.noreply.github.com>

nytzuga requested review from anusha-ctrl, ceyonur, darioush and aaronbuchwald as code owners June 30, 2023 21:06

nytzuga force-pushed the ginkgo-tests-parallel branch 10 times, most recently from bee4a3a to 92e1866 Compare July 10, 2023 04:14

darioush reviewed Jul 10, 2023

View reviewed changes

tests/precompile/solidity/suites.go Outdated Show resolved Hide resolved

tests/precompile/solidity/suites.go Outdated Show resolved Hide resolved

tests/precompile/solidity/suites.go Outdated Show resolved Hide resolved

nytzuga force-pushed the ginkgo-tests-parallel branch 5 times, most recently from 4840b16 to abaee1b Compare July 11, 2023 18:29

nytzuga changed the title ~~WIP: Ginkgo tests parallel~~ Ginkgo tests parallel Jul 11, 2023

darioush reviewed Jul 11, 2023

View reviewed changes

nytzuga force-pushed the ginkgo-tests-parallel branch from e333e63 to 51b6f91 Compare July 11, 2023 19:23

darioush approved these changes Jul 11, 2023

View reviewed changes

aaronbuchwald reviewed Jul 13, 2023

View reviewed changes

tests/precompile/solidity/suites.go Outdated Show resolved Hide resolved

aaronbuchwald requested changes Jul 13, 2023

View reviewed changes

nytzuga force-pushed the ginkgo-tests-parallel branch 3 times, most recently from 10ef2b0 to 35812a6 Compare July 14, 2023 05:02

ceyonur reviewed Jul 14, 2023

View reviewed changes

nytzuga force-pushed the ginkgo-tests-parallel branch 5 times, most recently from a823ec2 to 863464e Compare July 14, 2023 17:31

nytzuga commented Jul 14, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

nytzuga force-pushed the ginkgo-tests-parallel branch from 863464e to 404994a Compare July 14, 2023 18:19

nytzuga commented Jul 14, 2023

View reviewed changes

nytzuga force-pushed the ginkgo-tests-parallel branch 2 times, most recently from 6ab7af1 to 9b79ff5 Compare July 14, 2023 18:33

aaronbuchwald reviewed Jul 14, 2023

View reviewed changes

Run ginkgo in parallel

bc65b73

Use SynchronizedBeforeSuite to create the node and all the subnet that are needed to run tests. Pass the blockchainID to each work processor and run the tests in parallel to reduce runtime

nytzuga force-pushed the ginkgo-tests-parallel branch from 9b79ff5 to bc65b73 Compare July 14, 2023 21:33

aaronbuchwald approved these changes Jul 14, 2023

View reviewed changes

darioush reviewed Jul 14, 2023

View reviewed changes

aaronbuchwald merged commit e23b01c into master Jul 14, 2023
7 checks passed

aaronbuchwald deleted the ginkgo-tests-parallel branch July 14, 2023 22:43

nytzuga added a commit that referenced this pull request Jul 17, 2023

Improve tests

1991b0b

Improve tests based on feedback from #712

nytzuga mentioned this pull request Jul 17, 2023

Improve tests #733

Merged

nytzuga added a commit that referenced this pull request Jul 17, 2023

Improve tests

dcf3c7f

Improve tests based on feedback from #712

nytzuga added a commit that referenced this pull request Jul 17, 2023

Improve tests

7ba0b94

Improve tests based on feedback from #712

aaronbuchwald pushed a commit that referenced this pull request Jul 18, 2023

Improve tests (#733)

8b2271d

Improve tests based on feedback from #712

aaronbuchwald mentioned this pull request Jul 24, 2023

Parallelize e2e tests #623

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ginkgo tests parallel #712

Ginkgo tests parallel #712

nytzuga commented Jun 30, 2023 •

edited

darioush left a comment •

edited

aaronbuchwald commented Jul 13, 2023

How this was tested

aaronbuchwald Jul 13, 2023

aaronbuchwald Jul 13, 2023

aaronbuchwald Jul 13, 2023

nytzuga Jul 14, 2023

ceyonur Jul 14, 2023

aaronbuchwald Jul 13, 2023

aaronbuchwald left a comment

ceyonur left a comment

ceyonur Jul 14, 2023

nytzuga Jul 14, 2023

ceyonur Jul 14, 2023

nytzuga Jul 14, 2023

ceyonur Jul 14, 2023

nytzuga Jul 14, 2023

marun Jul 15, 2023

ceyonur Jul 14, 2023

nytzuga Jul 14, 2023

aaronbuchwald Jul 14, 2023

nytzuga Jul 14, 2023

aaronbuchwald Jul 14, 2023

aaronbuchwald Jul 14, 2023

aaronbuchwald left a comment

darioush Jul 14, 2023

nytzuga Jul 15, 2023

darioush Jul 14, 2023

darioush Jul 14, 2023

darioush Jul 14, 2023

darioush Jul 14, 2023

darioush Jul 14, 2023

	// Our test suite runs in a separated processes, ginkgo hasI
	// Our test suite runs in separate processes, ginkgo has

	// SynchronizedBeforeSuite() which is promised to run once, and its output is
	// SynchronizedBeforeSuite() which is run once, and its return value is

	// In here an AvalancheGo node instance is started, and subnets are created for
	// Here an AvalancheGo node is started, and subnets are created for

		log.Info("Executing HardHat tests on a new blockchain", "blockchainID", blockchainID, "test", test)
		log.Info("Using subnet", "ChainURI", chainURI)

Ginkgo tests parallel #712

Ginkgo tests parallel #712

Conversation

nytzuga commented Jun 30, 2023 • edited

Why this should be merged

How this works

How this was tested

How is this documented

darioush left a comment • edited

Choose a reason for hiding this comment

aaronbuchwald commented Jul 13, 2023

How this was tested

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronbuchwald left a comment

Choose a reason for hiding this comment

ceyonur left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronbuchwald left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nytzuga commented Jun 30, 2023 •

edited

darioush left a comment •

edited