fix failing tests in PRs from other repos, fixes #459 #461

ewingrj · 2017-11-13T17:13:19Z

A short summary which serves as a squashed-commit message.

Fix failing tests for users w/o access to the ACCOUNT_PASSWORD env variable. And disable e2e public network tests by travis for pull requests. Exclude lib dir from unit tests

A description to understand introduced changes without reading the code.

Dynamically load the TestConfig.Account1 & TestConfig.Account2 objects depending on if the network is StatusChain or a public testnet.

Important changes:

travis -> disable e2e tests on public network for pull_requests
add 2 new test accounts to be used with unit-tests & e2e tests using
StatusChain
dynamically set the TestConfig.Account* data based on the current
network
exclude lib dir from UNIT_TEST_PACKAGES in Makefile
added 2 new keystore file & updated addresses in Status genesis block

Solves #459

ewingrj · 2017-11-13T23:04:07Z

geth/account/accounts_test.go

@@ -100,12 +99,11 @@ func TestVerifyAccountPasswordWithAccountBeforeEIP55(t *testing.T) {
 	defer os.RemoveAll(keyStoreDir) //nolint: errcheck

 	// Import keys and make sure one was created before EIP55 introduction.
-	err = common.ImportTestAccount(keyStoreDir, "test-account3-before-eip55.pk")
-	require.NoError(t, err)
+	require.NoError(t, common.ImportTestAccount(keyStoreDir, "test-account4-public-pass.pk"))


I'm not sure if this is correct, or if I need to generate a pre-eip55 keystore for this test.

ewingrj · 2017-11-13T23:27:11Z

so it looks like the unit tests failed, due to the change in the genesis config.

Locally I had to rm -rf .ethereumtest and then the tests ran successfully. Only .ethereumtest/StatusChain should need to be removed.

I'm not that familiar with travis, but it appears that the build cache is interfering. I'm not sure if that's b/c I updated this PR and opening a new PR would fix it, or if clearing the build cache is something an admin has to do.

adambabik · 2017-11-15T06:28:05Z

@perissology, the first thing, thank you for the PR! It's amazing to see how many people have got interested in contributing to Status after Devcon3.

Locally I had to rm -rf .ethereumtest and then the tests ran successfully. Only .ethereumtest/StatusChain should need to be removed.

I will do it when we decide on a solution.

I'm not that familiar with travis, but it appears that the build cache is interfering. I'm not sure if that's b/c I updated this PR and opening a new PR would fix it, or if clearing the build cache is something an admin has to do.

You're correct, only admins can clean up Travis cache for a specific project.

Regarding your solution, the PRs from forks won't execute e2e tests on the public network, so you don't need to change those tests. It's important to keep the old ones as they will have some ether to actually perform some transactions.

The only thing that is left is geth/account/accounts_test.go. I wonder if we can maybe dynamically create a new account in this test suite as a setup step. This account won't be used to perform any transactions so it does not need to be added to genesis block or anywhere else. What do you think?

ewingrj · 2017-11-15T15:23:54Z

Regarding your solution, the PRs from forks won't execute e2e tests on the public network, so you don't need to change those tests. It's important to keep the old ones as they will have some ether to actually perform some transactions.

I'm confused about this statement. I don't believe I changed any e2e test files. Did I?

The only thing that is left is geth/account/accounts_test.go. I wonder if we can maybe dynamically create a new account in this test suite as a setup step. This account won't be used to perform any transactions so it does not need to be added to genesis block or anywhere else. What do you think?

I think this would be a fine solution, but since lib/utils.go is run with the make test-unit-coverage cmd, I need to modify the genesis block anyways. Why not just use that new account?
Maybe I should dynamically generate the pre eip55 account only?

I also realized that I never ran the e2e tests on a private network, but are not fixed by this PR yet.

It looks like there will need to be some logic in the tests that retrieves the appropriate account to use depending on the current network. For the e2e tests on the public network, we'll need to use Account1 & Account2, as those address have eth. However we can't use those on the private network as we don't have the password available, so we need to use new accounts. So as I see it, we need to have a testing utilty function GetAccount1() and GetAccount2() that returns the appropriate account based on the current network.

lmk if that sounds correct and I'll implement it.

adambabik · 2017-11-15T17:30:13Z

I'm confused about this statement. I don't believe I changed any e2e test files. Did I?

Oh, sorry. lib/ is actually run during both unit and e2e tests: https://github.com/status-im/status-go/blob/develop/Makefile#L118 :/ It should be run only in e2e tests.

Maybe I should dynamically generate the pre eip55 account only?

This is needed only for that one test TestVerifyAccountPasswordWithAccountBeforeEIP55 to test if the app can properly import accounts created before EIP55.

Your proposed solution sounds good. Unit and e2e tests on a private network should be executed by anyone and the only solution I see is that we have two sets of accounts and some logic to select them based on the network.

ewingrj · 2017-11-16T15:28:57Z

okay, I've updated the PR.

I didn't dynamically generate the account for TestVerifyAccountPasswordWithAccountBeforeEIP55 however. Not being that familiar w/ status codebase, I'm not sure of the best way to do this. I thought about using geth/account/accounts.go CreateAccount however that requires a valid commons.NodeManager when creating the acctManager. It seems to me that doing that would make this a integration test instead of a unit-test.

My solution was to just update the Account3 keystore file and hard-code the password in test-data.json. I'm happy to dynamically generate the account if you have any suggestions.

adambabik

LGTM

adambabik · 2017-11-16T18:01:29Z

Makefile

@@ -37,7 +37,7 @@ HELP_FUN = \

 # Main targets

-UNIT_TEST_PACKAGES := $(shell go list ./...  | grep -v /vendor | grep -v /e2e | grep -v /cmd)
+UNIT_TEST_PACKAGES := $(shell go list ./...  | grep -v /vendor | grep -v /e2e | grep -v /cmd | grep -v /lib)


adambabik · 2017-11-16T18:03:53Z

geth/common/types.go

-	testConfig.Account2.Password = pass
-	testConfig.Account3.Password = pass
+	if networkId == params.StatusChainNetworkID {
+		accountsData := string(static.MustAsset("config/status-chain-accounts.json"))


This is maybe better because at least consistent but just FYI if you omit conversion to string, you can omit conversion to []byte in the next line.

adambabik · 2017-11-16T18:12:04Z

static/config/status-chain-genesis.json

@@ -6,10 +6,10 @@
     "65C01586aa0Ce152835c788aCe665e91Ab3527b8": {
      "balance": "0x1B1AE4D6E2EF500000"
    },
-     "F35E0325dad87e2661c4eF951d58727e6d583d5c": {
+     "bF164ca341326a03b547c05B343b2E21eFAe24b9": {


I wonder if two accounts above are needed anymore...?

It looks like the 1st account is used in rpc_test.go, but just to check that eth_getBalance call returns > 0. This could be replaced w/ 1 of the new accounts.

The 2nd account doesn't appear to be used.

Oh, because it's hardcoded. Let's leave it for now then.

We don't need them. As I remember, that balance check used to be sort of a sanity check ensuring we have enough balance on the accounts which we use to send transactions.

adambabik · 2017-11-16T18:15:16Z

testing/testing.go

@@ -152,7 +152,7 @@ func GetHeadHashFromNetworkID(id int) string {
 	case params.RopstenNetworkID:
 		return "0x41941023680923e0fe4d74a34bdac8141f2540e3ae90623718e47d66d1ca4a2d"
 	case params.StatusChainNetworkID:
-		return "0x50e6edb4e90d9616ac8bf7119ee37c4048c41ad09328676e1b39dc68a0ecfb3d"
+		return "0xa286ebaba3b5c6ea836d1ffdef9d7573b0f5ccdfe2ebc51fa432fb9c58fb9774"


How did you figure out that? Just by getting the first block of the chain?

Just by getting the first block of the chain

essentially yes, I just let the tests do it for me...

there was a failing e2e/api test after changing the genesis file. 0xa286 was the returned hash in the failing test.

adambabik · 2017-11-16T18:34:50Z

My solution was to just update the Account3 keystore file and hard-code the password in test-data.json. I'm happy to dynamically generate the account if you have any suggestions.

No need, what you did looks great. The most important is that before-EIP55 account in the json file is lowercase. That's enough to test it.

I run e2e tests on a private chain locally and they passed. I will try to play with Travis cache to pass it here as well.

andytudhope · 2017-11-16T18:46:45Z

Hey @perissology while we wait for Travis, thanks again from the community for getting involved! Would you like to join chat.status.im? Would love to talk more about GivEth and how we can work closer together as teams in particular, and also just shoot the crypto breeze when you're not writing good code 😉

ewingrj · 2017-11-16T19:04:26Z

@andytudhope I'm already on the chat. Definitely interested in how we (Giveth) can collaborate more. I know Griff has talked to some of your team in regards to closer collaboration, but not sure of the details

adambabik · 2017-11-16T20:04:56Z

@perissology Hooray! e2e tests on a private chain passed. However, Travis also tried to run e2e tests on a public chain :( The condition in Travis should be a bit different I think. Maybe if: !fork?

This commit fixes issue status-im#459 - travis -> disable e2e tests on public network for pull_requests - add 2 new test accounts to be used with unit-tests & e2e tests using StatusChain - dynamically set the TestConfig.Account* data based on the current network - exclude lib dir from UNIT_TEST_PACKAGES in Makefile

ewingrj · 2017-11-16T21:25:20Z

okay, sorry for spamming travis :) it looks like I got it figured out now. type != pull_request.

adambabik · 2017-11-16T23:09:26Z

.travis.yml

@@ -17,6 +17,7 @@ jobs:
  - stage: Test e2e on privite network
    script: make test-e2e
  - stage: Test e2e on public network
+    if: type != pull_request


Another solution would be to use if: fork == false (there is such an attribute). PRs from the origin would still run this step as the env variables will be passed.

I did try if: fork == false and they still ran. That build is at https://travis-ci.org/status-im/status-go/jobs/303220190/config

Interesting, it looks like a bug. Let's keep type != pull_request then 👍

Add a comment justifying this line, please. It's non-obvious.

adambabik · 2017-11-16T23:16:36Z

@perissology no worries :) So, I have just one final comment about using a fork attribute instead of disabling that last step for all PRs. However, using fork will introduce inconsistency which we'd like to also avoid...

I guess we will be fine with your current approach (using if: type != pull_request) and I hope that in the next week I will be able to push forward #422 which will better define which tests should run when.

@tiabc I think this PR is good to be merged unless you have some comments :)

tiabc · 2017-11-17T03:53:33Z

testing/testing.go

+
+// GetAccount1PKFile returns the filename for Account1 keystore based
+// on the current network
+func GetAccount1PKFile() string {


I think having an intention as a comment would be more useful for understanding why it's needed.

tiabc · 2017-11-17T03:54:35Z

static/config/test-data.json

@@ -11,6 +11,7 @@
    "Address": "0xA0a19221268d939c3a972bf5461dC10f7980E814"
  },
  "Account3": {
-    "Address": "0xa7e0214f35481b644e139d689db54b3e625f1c88"
+    "Address": "0x3ad34e698d4806afd08b359b920f5c6b62b68ee4",
+    "Password": "password"


I suppose password shouldn't be here?

I need to store the password somewhere so geth/account/accounts_test.go TestVerifyAccountPasswordWithAccountBeforeEIP55 can run w/o access to the ACCOUNT_PASSWORD env variable. It seemed like the json file was a better place to store the password then hardcoding it in a function.

Yeah but what I mean is that it had better be in status-chain-accounts.json right? By following the logic that accounts in that file are supposed to be run with a hardcoded password while these should run with a password from an env variable.

ah, I see. If I do that, then TestConfig.Account3 will be nil when running tests on the public chain. I don't think that will break anything, as only the unit tests use Account3.

If that's okay, then maybe I should pull the accounts out of that file entirely. Place Account1 & Account2 in a public-chain-accounts.json & Account3 in status-chain-accounts.json. That feels like a cleaner solution then having some accounts in test-data.json and then overriding those accounts when on StatusChain.

Sounds good. Please, do.

tiabc

Thanks for the contribution! All seems good except for a couple of minor comments.

move public testnet accounts from test-data.json to a public-chain-accounts.json file. Remove 2 unnecessary account allocations in the status-chain-genesis file

ewingrj · 2017-11-18T04:19:14Z

okay, I've integrated all the comments. Probably need to reset the build cache to get travis to pass as I modified the genesis file again, removing the 2 accounts mentioned above

ewingrj force-pushed the develop branch from ddae076 to 54ceca0 Compare November 13, 2017 23:02

ewingrj commented Nov 13, 2017

View reviewed changes

ewingrj changed the title ~~[WIP] fix failing tests in PRs from other repos~~ fix failing tests in PRs from other repos Nov 14, 2017

ewingrj force-pushed the develop branch from 54ceca0 to bf790f8 Compare November 16, 2017 15:28

adambabik approved these changes Nov 16, 2017

View reviewed changes

ewingrj force-pushed the develop branch 3 times, most recently from 3bf7cd8 to e5af63b Compare November 16, 2017 20:57

ewingrj force-pushed the develop branch from e5af63b to d829a95 Compare November 16, 2017 21:22

adambabik reviewed Nov 16, 2017

View reviewed changes

monokh mentioned this pull request Nov 16, 2017

Improve JSON-RPC handling for raw requests/responses, Fixes #364 #458

Merged

3 tasks

tiabc reviewed Nov 17, 2017

View reviewed changes

tiabc approved these changes Nov 17, 2017

View reviewed changes

tiabc changed the title ~~fix failing tests in PRs from other repos~~ fix failing tests in PRs from other repos, fixes #459 Nov 17, 2017

ewingrj added 2 commits November 17, 2017 06:21

added comments

46b8f8c

separate test accounts into seperate files

07df5eb

move public testnet accounts from test-data.json to a public-chain-accounts.json file. Remove 2 unnecessary account allocations in the status-chain-genesis file

adambabik merged commit f0beeb3 into status-im:develop Nov 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix failing tests in PRs from other repos, fixes #459 #461

fix failing tests in PRs from other repos, fixes #459 #461

ewingrj commented Nov 13, 2017 •

edited

Loading

ewingrj Nov 13, 2017

ewingrj commented Nov 13, 2017 •

edited

Loading

adambabik commented Nov 15, 2017 •

edited

Loading

ewingrj commented Nov 15, 2017

adambabik commented Nov 15, 2017

ewingrj commented Nov 16, 2017

adambabik left a comment

adambabik Nov 16, 2017

adambabik Nov 16, 2017

adambabik Nov 16, 2017

ewingrj Nov 16, 2017

adambabik Nov 16, 2017

tiabc Nov 17, 2017

adambabik Nov 16, 2017

ewingrj Nov 16, 2017 •

edited

Loading

adambabik commented Nov 16, 2017

andytudhope commented Nov 16, 2017 •

edited

Loading

ewingrj commented Nov 16, 2017

adambabik commented Nov 16, 2017

ewingrj commented Nov 16, 2017

adambabik Nov 16, 2017

ewingrj Nov 16, 2017

adambabik Nov 17, 2017

tiabc Nov 17, 2017

adambabik commented Nov 16, 2017

tiabc Nov 17, 2017

tiabc Nov 17, 2017

ewingrj Nov 17, 2017

tiabc Nov 17, 2017

ewingrj Nov 17, 2017 •

edited

Loading

tiabc Nov 18, 2017

tiabc left a comment

ewingrj commented Nov 18, 2017

fix failing tests in PRs from other repos, fixes #459 #461

fix failing tests in PRs from other repos, fixes #459 #461

Conversation

ewingrj commented Nov 13, 2017 • edited Loading

Choose a reason for hiding this comment

ewingrj commented Nov 13, 2017 • edited Loading

adambabik commented Nov 15, 2017 • edited Loading

ewingrj commented Nov 15, 2017

adambabik commented Nov 15, 2017

ewingrj commented Nov 16, 2017

adambabik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ewingrj Nov 16, 2017 • edited Loading

Choose a reason for hiding this comment

adambabik commented Nov 16, 2017

andytudhope commented Nov 16, 2017 • edited Loading

ewingrj commented Nov 16, 2017

adambabik commented Nov 16, 2017

ewingrj commented Nov 16, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adambabik commented Nov 16, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ewingrj Nov 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tiabc left a comment

Choose a reason for hiding this comment

ewingrj commented Nov 18, 2017

ewingrj commented Nov 13, 2017 •

edited

Loading

ewingrj commented Nov 13, 2017 •

edited

Loading

adambabik commented Nov 15, 2017 •

edited

Loading

ewingrj Nov 16, 2017 •

edited

Loading

andytudhope commented Nov 16, 2017 •

edited

Loading

ewingrj Nov 17, 2017 •

edited

Loading