Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC0684: Introducing E2E testing in React Native core #684

Merged
merged 10 commits into from Aug 2, 2023
Merged

RFC0684: Introducing E2E testing in React Native core #684

merged 10 commits into from Aug 2, 2023

Conversation

kelset
Copy link
Member

@kelset kelset commented Jul 6, 2023 •
edited

Summary

Testing is how we validate that the code is good, and ensure that what we build our products on top of is solid and our customers are happy.

For the longest time, in React Native core this validation has been done in a few ways:

  • via Meta using the source code in their own products first
  • via internal testing with custom tooling in the Meta monorepo
  • via (mostly) unit testing on the CircleCI GitHub react-native repo
  • via local testing by the release crew ahead of generating new releases
  • via "pre-stable" releases (AKA Release Candidates and nightlies) that members from the community could use to test and report back

There is a great opportunity for improvement here, by introducing End-to-End (E2E) testing in the CircleCI GitHub react-native repository to introduce one more level of validation that aims to replace the need for (at least) the local testing from the release crew. By adding tooling that spins up the RNTester app in the codebase, and "black box" (the RNTester app is not aware that the agent using it is not a human) testing it, we can trust that the code is more solid.

This will make the codebase more stable, surface issues rapidly to the commit authors, and reduce the workload for the release crew.

Click here to view the rendered RFC

@kelset kelset added the 💡 Proposal This label identifies a proposal label Jul 6, 2023
@kelset kelset changed the title add rfc - starting point RFC0684: Introducing E2E testing in React Native core Jul 6, 2023
@kelset kelset changed the title RFC0684: Introducing E2E testing in React Native core RFC0684: Introducing E2E testing in React Native core (WIP) Jul 6, 2023
@kelset kelset changed the title RFC0684: Introducing E2E testing in React Native core (WIP) RFC0684: Introducing E2E testing in React Native core Jul 11, 2023
@kelset kelset marked this pull request as ready for review July 19, 2023 10:05
@kelset kelset mentioned this pull request Jul 19, 2023
Closed
szymonrybczak
szymonrybczak approved these changes Jul 24, 2023
View reviewed changes
Copy link
Contributor

@szymonrybczak szymonrybczak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

cortinico
cortinico approved these changes Jul 25, 2023
View reviewed changes
Copy link
Member

@cortinico cortinico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM mostly nits

proposals/0684-adding-e2e-in-core.md Outdated Show resolved Hide resolved
proposals/0684-adding-e2e-in-core.md

Because this proposal doesn't directly affect consumers of React Native, the need for "teaching content" is scoped to people who wants to learn more about this infrastructure is set up and they want to contribute with more tests.

The first place where this type of documentation will be added is the readme of the dedicated new folder, `packages/rn-tester-e2e/README.md` *(you can already check out a draft of it [in the first PR](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md) already in the works)*. In this file, there are going to be information on how the setup works, how to test it locally and how to add new tests.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO Let's update the link once the PR is merged

proposals/0684-adding-e2e-in-core.md Outdated Show resolved Hide resolved
facebook-github-bot pushed a commit to facebook/react-native that referenced this pull request Jul 26, 2023
@kelset @facebook-github-bot
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
3c196fb 
…#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](#36267 (comment))
* [a small refactoring into a js script](#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: #36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D46225747

Pulled By: cortinico

fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6
facebook-github-bot pushed a commit to facebook/react-native that referenced this pull request Jul 26, 2023
@kelset @facebook-github-bot
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
3c6dbec 
…#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](#36267 (comment))
* [a small refactoring into a js script](#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: #36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D47763012

Pulled By: cortinico

fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5
billnbell pushed a commit to billnbell/react-native that referenced this pull request Jul 29, 2023
@kelset
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
a4e79ea 
…facebook#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](facebook#36267 (comment))
* [a small refactoring into a js script](facebook#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: facebook#36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D46225747

Pulled By: cortinico

fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6
billnbell pushed a commit to billnbell/react-native that referenced this pull request Jul 29, 2023
@kelset
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
790bb82 
…facebook#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](facebook#36267 (comment))
* [a small refactoring into a js script](facebook#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: facebook#36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D47763012

Pulled By: cortinico

fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5
billnbell pushed a commit to billnbell/react-native that referenced this pull request Jul 29, 2023
@kelset
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
93ec9d1 
…facebook#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](facebook#36267 (comment))
* [a small refactoring into a js script](facebook#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: facebook#36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D46225747

Pulled By: cortinico

fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6
billnbell pushed a commit to billnbell/react-native that referenced this pull request Jul 29, 2023
@kelset
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
c597d33 
…facebook#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](facebook#36267 (comment))
* [a small refactoring into a js script](facebook#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: facebook#36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D47763012

Pulled By: cortinico

fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5
billnbell pushed a commit to billnbell/react-native that referenced this pull request Jul 29, 2023
@kelset
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
a7fde6c 
…facebook#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](facebook#36267 (comment))
* [a small refactoring into a js script](facebook#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: facebook#36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D46225747

Pulled By: cortinico

fbshipit-source-id: e5428c439afd84ed3dd219d6ada538263120d7c6
billnbell pushed a commit to billnbell/react-native that referenced this pull request Jul 29, 2023
@kelset
add RNTester-E2E: tests for iOS and Android via Appium, WDIO and Jest (…
22e6f16 
…facebook#36267)

Summary:
The motivation is to create an E2E testing solution for the RNTester app that can be used to flag when things break and the release crew can use to validate more of the codebase ahead of a release.

This first PR wants to just showcase the main flow (how tests are made, where the E2E folder lives, how it's used by CircleCI) and then we can build on top of it with subsequent PRs, with the help of an umbrella issue to get the community involved in making more tests!

This work is being done cooperation with Callstack developers (mateuszm22 , adzironman, and others) - reason why the branch [lives in a fork](https://github.com/mateuszm22/react-native/tree/k%2Bm/new-rn-tester-E2E).

### [Read the RFC for more details ](react-native-community/discussions-and-proposals#684)

### Extra notes

We are still on WebDriverIO 7 because during the exploratory phase, WDIO was a bit problematic and the workarounds needed to make it work don't seem very nice - see this PR: kelset/react-native-e2e-jest-appium-webdriverio#4 --- this will be revisited in the future, once there's a better path for upgrading.

### Things that will be handled as follow up work

* upgrade to WDIO 8
* [an automated yarn run-e2e-test script](facebook#36267 (comment))
* [a small refactoring into a js script](facebook#36267 (comment))

## Changelog:

[INTERNAL] [ADDED] - Added first working configuration for e2e testing

Pull Request resolved: facebook#36267

Test Plan:
You can play with this locally by [follow the readme](https://github.com/mateuszm22/react-native/blob/k+m/new-rn-tester-E2E/packages/rn-tester-e2e/README.md).

CircleCI jobs will be added.

Reviewed By: cipolleschi

Differential Revision: D47763012

Pulled By: cortinico

fbshipit-source-id: 6eb9674182b8ee97aea4784158c69bf017f379e5
@kelset @szymonrybczak
Update proposals/0684-adding-e2e-in-core.md
760c4e0 
Co-authored-by: Szymon Rybczak <szymon.rybczak@gmail.com>
@cortinico cortinico merged commit d907dfa into main Aug 2, 2023
@cortinico cortinico deleted the kelset/rfc-e2e-testing branch August 2, 2023 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@szymonrybczak szymonrybczak szymonrybczak approved these changes

@cortinico cortinico cortinico approved these changes

@cipolleschi cipolleschi cipolleschi approved these changes

Assignees
No one assigned
Labels
💡 Proposal This label identifies a proposal
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@kelset @cortinico @cipolleschi @szymonrybczak