Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(release): Update release testing section for new E2E testing script #3368

Merged
merged 6 commits into from
Oct 18, 2022
Merged

chore(release): Update release testing section for new E2E testing script #3368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
eb809cd
remove E2E section until we'll re-add it post E2E setup
kelset Oct 10, 2022
47ec4ca
remove mention of the E2E script
kelset Oct 10, 2022
bfc1f85
rework the release testing page
kelset Oct 10, 2022
2a585c4
add mention of where to find the info to turn on new arch
kelset Oct 10, 2022
4d60d81
typo
kelset Oct 14, 2022
9aba10a
re-add section
kelset Oct 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 7 additions & 7 deletions website/contributing/how-to-contribute-code.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@ Please refer to the [Environment Setup](/docs/environment-setup) guide to setup

After cloning React Native, open the directory and run `yarn` to install its dependencies.

Then, you can run several commands:
Now you are set up to run several commands:

- `yarn start` starts the Metro packager server.
- `yarn lint` checks the code style.
- `yarn format` automatically formats your code.
- `yarn test` runs the JavaScript test suite.
- `yarn test --watch` runs an interactive JavaScript test watcher.
- `yarn test <pattern>` runs JavaScript tests with matching filenames.
- `yarn test` runs the Jest-based JavaScript test suite.
- `yarn test --watch` runs an interactive JavaScript test watcher.
- `yarn test <pattern>` runs JavaScript tests with matching filenames.
- `yarn flow` runs the [Flow](https://flowtype.org/) typechecks.
- `yarn flow-check-android` does a full Flow check over `*.android.js` files.
- `yarn flow-check-ios` does a full Flow check over `*.ios.js` files.
- `yarn flow-check-android` does a full Flow check over `*.android.js` files.
- `yarn flow-check-ios` does a full Flow check over `*.ios.js` files.
- `yarn test-typescript` runs the [TypeScript](https://www.typescriptlang.org/) typechecks.
- `yarn test-ios` runs the iOS test suite (macOS required).
- `node ./scripts/run-ci-e2e-tests.js --js --android --ios` runs the end-to-tend JavaScript, Android, and iOS tests.

## Testing your Changes

Expand Down
11 changes: 1 addition & 10 deletions website/contributing/how-to-run-and-write-tests.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ Xcode also allows running individual tests through its Test Navigator. You can a
If you are making modifications to `Podfile` configurations then there are Ruby tests that can verify these.

To run the ruby tests:

```bash
cd scripts
sh run_ruby_tests.sh
Expand Down Expand Up @@ -79,16 +80,6 @@ Then, run the Android integration tests:
./scripts/run-android-local-integration-tests.sh
```

### End-to-end Tests

Finally, make sure end-to-end tests run successfully by executing the following script:

```bash
./scripts/test-manual-e2e.sh
```

This script will make you select to run the RNTester app (that lives within `packages/rn-tester`) or it will generate a fresh new project in `/tmp/RNTestProject`. Follow the step-by-step instructions from the script to test successfully if your changes.

## Writing Tests

Whenever you are fixing a bug or adding new functionality to React Native, it is a good idea to add a test that covers it. Depending on the change you're making, there are different types of tests that may be appropriate.
Expand Down
112 changes: 61 additions & 51 deletions website/contributing/release-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,20 @@ id: release-testing
title: How to Test a Release
---

## Test source in the release branch

These steps needs to be followed by the release crew as part of the release process, to ensure that new versions published have a good level of stability.

### Pre-requisites
:::info

An important element of testing source in the release branch is that this process needs to be preferably be done twice, on two separate machines - there are multiple scenarios in which something might fail on a machine but not on another. By running tests on two computers, we want to reach a step of double confirmation that everything works fine.
:::

## Pre-requisites

:::note

Currently, this flow can only be done on macOS machines.

:::

- Have a clone of `react-native` repo and be on the release branch (`0.XX-stable`).

Expand All @@ -19,14 +28,18 @@ These steps needs to be followed by the release crew as part of the release proc
echo '{}' > .watchmanconfig
```

- Have Android and iOS development environment set-up. Follow instructions for macOS/iOS and macOS/Android from the [Environment Setup](/docs/environment-setup) guide.
- Have Android and iOS development environment set-up. Follow instructions for `React Native CLI quickstart` for macOS/iOS and macOS/Android from the [Environment Setup](/docs/environment-setup) guide.

#### Additional steps for Android

- Gradle should now install [the appropriate NDK](https://github.com/facebook/react-native/blob/main/template/android/build.gradle). Verify that you have in your path the `ANDROID_NDK` variable, pointing to it.
- In case you are on macOS Catalina (or higher), you might also need to run `sudo xattr -r -d com.apple.quarantine /path/to/ndk` to avoid the e2e script to fail. (_That said, this should not happen anymore since from NDK 21 and higher the Android team started signing the NDK._)

### Steps
## Steps

### Clean up the local state

When testing locally, we want to ensure that we start from a clean slate to avoid caches polluting our testing.

1. Delete `RNTester` and `RNTestProject` from your Android emulator and iOS simulator if leftover from previous test.
2. Remove any temporary files from the `react-native` repo:
Expand All @@ -35,76 +48,73 @@ These steps needs to be followed by the release crew as part of the release proc
git clean -fdx
```

For `main` branch, and versions of RN >=0.71, you can instead use `yarn test-e2e-local-clean`.

3. Install dependencies:

```bash
yarn install
pushd packages/rn-tester
pod install --repo-update
popd
```

4. Use the `test-manual-e2e` script to test `RNTester` and the template app (`RNTestProject`):
### Generating the projects

```bash
# This will run you through the different variants in Test Dimensions table
./scripts/test-manual-e2e.sh
```
The local testing for a release consist of running the [test project](https://github.com/facebook/react-native/tree/main/packages/rn-tester) of the react-native repository, `RNTester`, which contains an in-depth list of components implementations, and generating a fresh new project based on the local code, `RNTestProject`, that will simulate accurately how a `react-native init` project will behave.

5. Turn on Hermes in the `RNTestProject` and ensures it builds successfully.
To generate the the right project with the specific configuration desired, you can use the command

- Enable Hermes for Android:
```bash
yarn test-e2e-local [options]
```

```bash
# Update `/tmp/RNTestProject/android/app/build.gradle` to `enableHermes`
project.ext.react = [
enableHermes: true, // clean and rebuild if changing
]
Followed by the wanted options:

# Clean and rebuild
/tmp/RNTestProject/android$ ./gradlew clean
/tmp/RNTestProject$ npx react-native run-android
```
```bash
--help Show help [boolean]
--version Show version number [boolean]
-t, --target [choices: "RNTester", "RNTestProject"] [default: "RNTester"]
-p, --platform [choices: "iOS", "Android"] [default: "iOS"]
-h, --hermes [boolean] [default: true]
```

- Enable Hermes for iOS:
#### Versions older than 71

```bash
# Update `/tmp/RNTestProject/ios/Podfile` and then run `pod install`
use_react_native!(
:path => config[:reactNativePath],
# to enable hermes on iOS, change `false` to `true` and then install pods
:hermes_enabled => true
)
You need to use the interactive script run you through the different variants in [Test Dimensions](#test-dimensions):

# Install pods and run
/tmp/RNTestProject/ios$ pod install
/tmp/RNTestProject$ npx react-native run-ios
```
```bash
./scripts/test-manual-e2e.sh
```

### Test Dimensions
This script will ask you to select which platform and which project you want to test, and then to execute a series of extra steps during the process. Bear in mind, when testing RNTester on Android, you need to start the Android emulator ahead of time or it will fail.

Covered by running `test-manual-e2e.sh`, see [issue](https://github.com/facebook/react-native/issues/33015) about supporting those "manual" cases.
## What to test?

| Variant | RNTester | Template App |
| ---------------- | ------------------------ | ------------------------ |
| Android - JSC | via `test-manual-e2e.sh` | via `test-manual-e2e.sh` |
| Android - Hermes | via `test-manual-e2e.sh` | manual |
| iOS - JSC | via `test-manual-e2e.sh` | via `test-manual-e2e.sh` |
| iOS - Hermes | via `test-manual-e2e.sh` | manual |
Aside from verifying that the building process is successful, once the app spawn by the script is up and running, we want to run a series of manual tests to ensure that some core functionalities work, like Fast Refresh and the Flipper debugger.

**Note well:** Starting from RN 0.70, Hermes is turned on by default so for the template app JSC will need to be manually tested by switching off Hermes.
In the `RNTester` you want to also play around with the app and try different components: some important onces are `Flatlist`, `Image` and the "New Architecture Component" (should be the very last one in the list).

### What to test?
### Test Dimensions

Aside from verifying that the building process is successful, once the app spawn by `test-manual-e2e.sh` is up and running, we want to run a series of manual tests to ensure that some core functionalities work, like Fast Refresh and the Flipper debugger.
To ensure that we cover the most use cases, we need to ensure we test all these different combination of configurations:

In the `RNTester` you want to also play around with the app and try different components: some important onces are `Flatlist`, `Image` and the "New Architecture Component" (should be the very last one in the list).
- RNTester + iOS + Hermes
- RNTester + iOS + JSC
- RNTester + Android + Hermes
- RNTester + Android + JSC
- RNTestProject + iOS + Hermes
- RNTestProject + iOS + JSC
- RNTestProject + Android + Hermes
- RNTestProject + Android + JSC

:::note

Bear in mind that RNTester project is already onboarded in the new architecture. `RNTestProject` is not - new architecture mode needs to be [enabled](/docs/the-new-architecture/use-app-template#enable-the-new-architecture) and tested separately.
:::

An important element of testing source in the release branch is that this process needs to be preferably be done twice, on two separate machines - there are multiple scenarios in which something might fail on a machine but not on another. By running tests on two computers, we want to reach a step of "double confirmation" that everything works fine.
## Testing pre-releases (RC) on production apps

## Testing an RC on a project - checklist
During the Release Candidate (RC) phase of a release cycle, we ask for the community to set as dependency in their apps the latest RC available and report in the related "Road to 0.XX" how it performs ([example](https://github.com/reactwg/react-native-releases/discussions/26)).

If you are a [release tester](./release-roles-responsibilities#release-tester-responsibilities), the ask for you is to set as dependency in your app the latest RC available and report in the related "Road to 0.XX" how it went ([example](https://github.com/reactwg/react-native-releases/discussions/26)). To help provide the relevant information, we have prepared this template: you can copy/pasted it in a comment and fill it accordingly.
To help provide the relevant information, we have prepared this template they can use as blueprint for what is important to test - they can copy/pasted it in a comment and fill it accordingly.

```markdown
| Link to branch: | |
Expand Down