Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reorganize test setup #1132

Merged
merged 26 commits into from Jul 16, 2019
Merged

Reorganize test setup #1132

merged 26 commits into from Jul 16, 2019

Conversation

Stefie
Copy link
Contributor

@Stefie Stefie commented Jul 15, 2019 •
edited

Addresses the following points of #908:

  • Reorder and better categorize existing tests
  • Move all e2e tests to ./api

I'm working through this part bit by by now for all supported versions

  • Fix broken / outdated tests

Also in this PR:
I split off the e2e test flags to be able to run the tests from the 3 different sources independently.
yarn test:e2e: Will run the e2e tests with the local node on 9944
yarn test:e2e-docker: Will run the e2e tests with the 5 Docker images
yarn test:e2e-remote: Will run the e2e tests with the 2 remote endpoints

The goal is that in the end everyone developing on the API can spot failing tests on all supported substrate and polkadot versions easily.
Screenshot 2019-07-15 at 18 18 08

Stefie added 16 commits July 12, 2019 17:45
@Stefie
Moved all e2e tests to 'api' folder
63aa15c
@Stefie
Moved metadata test
c9f7dad
@Stefie
Merge branch 'master' into sd-reorganize-test-setup
7771577 
Conflicts:
	packages/api/test/e2e/api-derive/promise.spec.ts
	packages/api/test/e2e/api-metadata/transfer.spec.ts
	packages/api/test/e2e/api/promise-consts.spec.ts
	packages/api/test/e2e/api/promise-contract.spec.ts
	packages/api/test/e2e/api/promise-queries.spec.ts
	packages/api/test/e2e/api/promise-tx.spec.ts
	packages/api/test/e2e/api/rx-queries.spec.ts
	packages/api/test/e2e/api/rx-tx.spec.ts
	packages/api/test/e2e/rpc-core/alexander.spec.ts
	packages/api/test/e2e/rpc-core/basics.spec.ts
	packages/api/test/e2e/rpc-core/chain.spec.ts
	packages/api/test/e2e/rpc-core/state.spec.ts
	packages/api/test/e2e/rpc-core/subscribe.spec.ts
@Stefie
fix imports
6a01d77
@Stefie
Merge branch 'master' into sd-reorganize-test-setup
911db8c 
Conflicts:
	packages/api/test/e2e/api/promise-alex-archive.spec.ts
	packages/api/test/e2e/api/promise-alex.spec.ts
	packages/api/test/e2e/api/promise-contract.spec.ts
	packages/api/test/e2e/api/promise-queries.spec.ts
	packages/api/test/e2e/api/promise-tx.spec.ts
	packages/api/test/e2e/api/rx-queries.spec.ts
	packages/api/test/e2e/api/rx-tx.spec.ts
@Stefie
Add jest options for testing the different endpoint types separately
704a162
@Stefie
Add check for empty test suites and return empt test to prevent jest …
f0a99e5 
…from throwing an error
@Stefie
Fix import paths
303ac3f
@Stefie
Fix const tests for polkadot chains
6c7d3d0
@Stefie
Update to latest Substrate 2.0 image
fdbb5ff
@Stefie
Move contract files for e2e testing to e2e tesing folder
8248677
@Stefie
reorder & fix promise query tests for all test nodes
6d4920b
@Stefie
Merge branch 'master' into sd-reorganize-test-setup
0b0be01
@Stefie
final fixes for promise query
93dfec5
@Stefie
eslint fixes
86fbad1
@Stefie
removed trailing space
2eb566e
@Stefie Stefie mentioned this pull request Jul 15, 2019
Closed
10 tasks
@Stefie Stefie requested a review from amaury1093 July 16, 2019 06:01
amaury1093
amaury1093 reviewed Jul 16, 2019
View reviewed changes
Copy link
Contributor

@amaury1093 amaury1093 left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all tests pass?

  • test:e2e (on local node): 6 failed
  • test:e2e-remote: 13 failed
  • test:e2e-docker: 23 failed

packages/api/test/e2e/api-derive/promise.spec.ts Outdated Show resolved Hide resolved
packages/api/test/e2e/api-derive/promise.spec.ts Outdated Show resolved Hide resolved
packages/api/test/e2e/api/promise-queries.spec.ts Outdated Show resolved Hide resolved
packages/api/test/util/describeE2E.ts Outdated Show resolved Hide resolved
packages/api/test/util/describeE2E.ts Outdated Show resolved Hide resolved
jacogr
jacogr reviewed Jul 16, 2019
View reviewed changes
Copy link
Member

@jacogr jacogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly sane, just 2 comments around splitting and contract locations.

packages/api/test/e2e/api/promise-contract.spec.ts Outdated Show resolved Hide resolved
packages/api/test/e2e/api/promise-queries.spec.ts Outdated Show resolved Hide resolved
@amaury1093
Copy link
Contributor

Would be cool to take this into account too: https://github.com/polkadot-js/api/blob/master/packages/api/test/e2e/promise-tx.spec.ts#L122

@Stefie
Copy link
Contributor Author

Stefie commented Jul 16, 2019 •
edited

Should all tests pass?

No, they shouldn't.
As I said in the PR description, I'm going through them bit by bit. I'd just like to merge this PR because I have another PR open for which I need the changes in here and I don't want to pull this one in and then keep both op to date with master and each other, because the file structure for the tests changed in this one.
And there will be more e2e tests passing after this PR got merged than before.

Files that should work are:
promise-alex.spec
promise-consts.spec
promise-queries-dev.spec.ts
promise-queries-doubleMap.spec.ts
promise-queries.spec.ts
rx-queries.spec.ts

@Stefie
Copy link
Contributor Author

Stefie commented Jul 16, 2019 •
edited

Would be cool to take this into account too: https://github.com/polkadot-js/api/blob/master/packages/api/test/e2e/promise-tx.spec.ts#L122

My current approach is:

  1. I'm going through all the e2e tests files and will make all tests pass or report bugs for all 5 supported versions
  2. refactor tests / add tests/ tackle TODO's

I'd take care of the promise-tx.spec file in one of my next PR's

@Stefie
Copy link
Contributor Author

Stefie commented Jul 16, 2019

Should all tests pass?

  • test:e2e (on local node): 6 failed

I generally won't care about the local tests passing btw because that's unpredictable. I could never know which version / node a person is running on their machine.
So for local tests I'll only make sure that the tests are running, but I don't care about the results.

amaury1093
amaury1093 approved these changes Jul 16, 2019
View reviewed changes
Copy link
Contributor

@amaury1093 amaury1093 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Stefie Stefie merged commit d57dca5 into master Jul 16, 2019
@Stefie Stefie deleted the sd-reorganize-test-setup branch July 16, 2019 14:03
ianhe8x pushed a commit to plugblockchain/api.js that referenced this pull request Jul 30, 2019
@Stefie
Reorganize test setup (polkadot-js#1132)
fb9e262 
* Moved all e2e tests to 'api' folder

* Moved metadata test

* fix imports

* Add jest options for testing the different endpoint types separately

* Add check for empty test suites and return empt test to prevent jest from throwing an error

* Fix import paths

* Fix const tests for polkadot chains

* Update to latest Substrate 2.0 image

* Move contract files for e2e testing to e2e tesing folder

* reorder & fix promise query tests for all test nodes

* final fixes for promise query

* eslint fixes

* removed trailing space

* fix linting errors

* Address codeclimate "Function getWsEndpoints has a Cognitive Complexity of 7 (exceeds 5 allowed)."

* Fix contract ABI import paths

* linter endpoint types

* replace alice with alice_stash

* split off promise-queries file

* Remove apiType, smaller cleanups, added local to only: [] tests suites

* relative paths changed to `@polkadot` import

* moved contract wasm & abi's back to `api-contract`
ianhe8x pushed a commit to plugblockchain/api.js that referenced this pull request Jul 31, 2019
@Stefie @ianhe8x
Reorganize test setup (polkadot-js#1132)
0258d43 
* Moved all e2e tests to 'api' folder

* Moved metadata test

* fix imports

* Add jest options for testing the different endpoint types separately

* Add check for empty test suites and return empt test to prevent jest from throwing an error

* Fix import paths

* Fix const tests for polkadot chains

* Update to latest Substrate 2.0 image

* Move contract files for e2e testing to e2e tesing folder

* reorder & fix promise query tests for all test nodes

* final fixes for promise query

* eslint fixes

* removed trailing space

* fix linting errors

* Address codeclimate "Function getWsEndpoints has a Cognitive Complexity of 7 (exceeds 5 allowed)."

* Fix contract ABI import paths

* linter endpoint types

* replace alice with alice_stash

* split off promise-queries file

* Remove apiType, smaller cleanups, added local to only: [] tests suites

* relative paths changed to `@polkadot` import

* moved contract wasm & abi's back to `api-contract`
@polkadot-js-bot
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@polkadot-js polkadot-js locked as resolved and limited conversation to collaborators Jun 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jacogr jacogr jacogr left review comments

@amaury1093 amaury1093 amaury1093 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@Stefie @amaury1093 @polkadot-js-bot @jacogr