Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: React Native WebAPIs #2504

Merged
merged 22 commits into from
Jan 12, 2024
Merged

RFC: React Native WebAPIs #2504

Changes from 16 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
226451d
React Native Standard API
tido64 Jul 4, 2023
d9d634c
add polyfill examples
tido64 Jul 5, 2023
0f4775e
add Lorenzo's first pass, some more in the adoption strategy, and a c…
kelset Jul 6, 2023
fed17d7
some more tweaks, add mention to monorepo and infra
kelset Jul 12, 2023
b706985
tweaks
tido64 Jul 18, 2023
98b0ea2
fixup! tweaks
tido64 Jul 18, 2023
e23792d
fixup! tweaks
tido64 Jul 18, 2023
2a04215
allow additional platform implementations
tido64 Aug 2, 2023
9bf372b
change the name from standard to webapi
kelset Aug 3, 2023
cb9f806
reflect proper npm org name
kelset Aug 3, 2023
8280738
Update text/0002-react-native-webapi.md
kelset Aug 3, 2023
5455b97
Update text/0002-react-native-webapi.md
kelset Aug 3, 2023
4508f3a
Update text/0002-react-native-webapi.md
kelset Aug 3, 2023
8409c71
Rename 0002-react-native-webapi.md to 0002-react-native-webapis.md
tido64 Aug 3, 2023
0567637
Update 0002-react-native-webapis.md
tido64 Aug 3, 2023
1c32d86
links to prototypes
tido64 Aug 8, 2023
714568b
mention Babel plugin
tido64 Aug 9, 2023
7baefbc
clarify how the plugin is used
tido64 Aug 9, 2023
18a3e1d
Battery Status API is deprecated
tido64 Aug 9, 2023
5684aba
feedback
tido64 Aug 24, 2023
cca0e1e
expand and update the RFC with new information
kelset Sep 1, 2023
df23ecb
small addition to RFC
kelset Sep 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
226 changes: 226 additions & 0 deletions text/0002-react-native-webapis.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
- Title: react-native-webapis
- Date: 2023-07-04
- RFC PR: https://github.com/microsoft/rnx-kit/pull/2504
- rnx-kit Issue: N/A

## Abstract

React Native currently lacks a well-defined, stable and complete API surface.
Compared to what both [Android](https://developer.android.com/reference) and
[iOS](https://developer.apple.com/documentation/technologies) provide out of the
box, React Native core is missing quite a lot today. To fill that gap, we have a
"wild west" of community modules, each with its own set of interfaces and
behaviors. While this is in line with the broader experience in the web/npm
space, at the end of the day this means that it is the developers'
responsibility to find modules that fit their needs and that are seemingly
actively maintained.

Additionally, the APIs are not compatible with Web APIs, thus closing the door
to a wealth of open source libraries that do not have explicit React Native
support. This often means that developers cannot reuse existing web code, and
must search for or even create one for their needs.

In this RFC, we are proposing to close this gap by providing our own
implementation of the
[Web APIs](https://developer.mozilla.org/en-US/docs/Web/API) for React Native.
The ultimate goal is to open the possibility to run non-UI code directly in a
React Native app; to provide a familiar environment for existing web developers
as well as a well-documented API for developers of any experience level.

## Guide-level explanation

We are aiming to have close to 100% compatibility with Web APIs. This means that
code such as below should just work out of box (or with minimal configuration).

```js
// useBatteryLevel.js
function useBatteryLevel() {
const [batteryLevel, setBatteryLevel] = useState(-1);
useEffect(() => {
navigator.getBattery().then((battery) => setBatteryLevel(battery.level));
}, [setBatteryLevel]);
return batteryLevel;
}
```

The native platforms we aim to support via react-native are Android, iOS, macOS,
Windows.

## Reference-level explanation

Our goal is not to reimplement a browser, e.g.
[Electron](https://www.electronjs.org/). Apps made this way currently ship with
a full browser with everything that entails, including modules that they may
never get used. Native apps, and mobile apps especially, cannot be shipped with
unused bits; nor does it make any sense to include MBs of dependencies that are
never used. Ideally, migrating from community modules to WebAPIs should not
increase the final app size (at least not significantly).

The API we envision are implemented in the following layers:

```mermaid
graph TD;
id(Web APIs)-->id1(JS shim);
id1(JS shim)-->id2(Turbo/native module);
id2(Turbo/native module)-->id3(Platform API/\nCommunity module);
tido64 marked this conversation as resolved.
Show resolved Hide resolved
```

- **Web APIs:** On the surface, there are little to no differences from typical
web code.

- **JS shim:** This is a thin layer for marshalling web API calls to native
modules. It needs to be installed before it can be used. The current thinking
is to install this layer as polyfills. Installing them on-demand is preferred,
but we need to start somewhere and we should be able to improve this layer
later without affecting users. In any case, we will need tools to include
polyfills similarly to [autolinking][].
kelset marked this conversation as resolved.
Show resolved Hide resolved

- **Turbo/native module:** This is the layer that accepts JS calls and forwards
them platform implementations. Modules are installed via the standard
autolinking mechanism.

tido64 marked this conversation as resolved.
Show resolved Hide resolved
- **Platform API/Community module:** When available, we want to be able to
leverage what already exists in the community and avoid creating competing
solutions. When we need to write from scratch, we want to comply with the
guidelines for the
[new architecture libraries](https://github.com/reactwg/react-native-new-architecture/discussions/categories/libraries)
provided by Meta.

### Modularity

We want to avoid introducing unused modules and adding unnecessary bloat to the
app bundle. WebAPIs must therefore be broken down into smaller modules that can
be installed separately. These modules are installed by autolinking, and must
therefore be explicitly added to an app's `package.json`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking about this too but the direction I was considering was a bit different. If we use the dependencies listed in package.json and the current autolinking approach, we might end up including JS and native code in the binary that's not actually used. The approach I was considering instead was doing 2 things:

  1. Use metadata generated by Metro during bundling to know what native modules are actually used in the code. Use that information for autolinking (so a module that's installed but not actually imported in the bundle would not be included).
  2. Use a Babel transform to automatically replace the browser API with imports to the native modules. E.g.:
// From
navigator.getBattery();

// To
require("@react-native-webapis/battery-status").getBattery();

Optionally, we could also make it so the resolved module is configurable by the user, so we can actually swap implementations whenever necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @rubennorte! Thanks for your feedback. I had some thoughts around these approaches as well (that I probably should put somewhere in the document),

  1. You're right that current native autolinking is too naive. Your proposal makes sense — the only thing I would add is that we probably still need to keep the current approach as fallback for when there is no bundle e.g., because a dev server is used. Somewhere in this document, I mentioned a tool for telling authors that they're using a Web API that has not been polyfilled and either suggest or add a reasonable library. This same tool should be able to produce the metadata you mentioned.
  2. Our current approach is to inject the polyfills at the beginning of the bundle. This approach simple to implement, but doesn't allow for tree shaking like yours does. Do you think replacing calls is something that we can perform with 100% accuracy? How do we know navigator doesn't refer to something else? I think the same problem applies to Range and other APIs? I'm not so well-versed in Babel or AST manipulation in general, so please forgive the dumb questions.

I'm thinking that we want to align with the behaviour of native autolinking i.e., we should allow autopolyfilling to be disabled via the very same mechanisms. If we can tell people that it works the exact same way native autolinking does, that will reduce cognitive load. We should aim to fix native autolinking behaviour as necessary. I'm a bit wary of introducing yet another concept and making it hard to debug/wrap your head around.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only thing I would add is that we probably still need to keep the current approach as fallback for when there is no bundle e.g., because a dev server is used.

Yeah, I think there are cases where forcing this is reasonable (including when using OTA), so we definitely need a way to force certain native modules to always be included regardless of whether they're used or not. We probably shouldn't be doing that by default though.

In the future, if we shipped the browser APIs with RN, we could have an option in the RN config like:

browserGlobals: {
  select?: 'detected' | 'all' | 'none',
  include?: Array<string>,  // doesn't work with 'all'
  exclude?: Array<string>, // doesn't work with 'none'
}

Somewhere in this document, I mentioned a tool for telling authors that they're using a Web API that has not been polyfilled and either suggest or add a reasonable library. This same tool should be able to produce the metadata you mentioned.

I think that tool would be useful but only in the case we discussed before (using OTA or SSR). For the metadata, my approach was about native modules, not about Web APIs specifically (which has the added benefit of reducing bloat in a broader set of cases).

Our current approach is to inject the polyfills at the beginning of the bundle. This approach simple to implement, but doesn't allow for tree shaking like yours does. Do you think replacing calls is something that we can perform with 100% accuracy? How do we know navigator doesn't refer to something else? I think the same problem applies to Range and other APIs? I'm not so well-versed in Babel or AST manipulation in general, so please forgive the dumb questions.

We can tell if the reference to navigator is global or if it's redefined/imported from another module. The only requirement from this would be that you use it directly (as in navigator.getBattery()) instead of using indirection (e.g.: const {getBattery} = navigator; getBattery();).

I'm thinking that we want to align with the behaviour of native autolinking i.e., we should allow autopolyfilling to be disabled via the very same mechanisms. If we can tell people that it works the exact same way native autolinking does, that will reduce cognitive load. We should aim to fix native autolinking behaviour as necessary. I'm a bit wary of introducing yet another concept and making it hard to debug/wrap your head around.

I think it's fair to focus on the current capabilities in RN to implement this, but we need to make sure it wouldn't conflict with the direction that we want to take in the long term.


For example, if you want to use `BatteryStatus` you should not need to import
the whole `react-native-webapis` module, but only the dedicated
`@react-native-webapis/battery-status` submodule.

Additionally, we want to avoid requiring that users manually add polyfills for
the modules they need. Instead, we propose that modules that implement a part of
the API to declare a polyfill in their `react-native.config.js`:

```js
// react-native.config.js
module.exports = {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tido64 could you use a specific example here? I'm not sure I understand the shape of this config 100%.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment, there really isn't much else to it. We're building on top of the existing autolinking schema as defined in the CLI: https://github.com/react-native-community/cli/blob/main/docs/autolinking.md#what-do-i-need-to-have-in-my-package-to-make-it-work

Just to be clear, this react-native.config.js lives in the module package. Not in the app project.

dependency: {
api: {
polyfill: "./polyfill.js",
},
},
};
```

Polyfills are gathered and passed to Metro. Any dependency with a correctly
Copy link

@rubennorte rubennorte Sep 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @motiz88. Flagging in case you have any concerns about this.

declared polyfill will be included. They do not need to be under the
`@react-native-webapis` scope or even live in the same repository.

### Infrastructure and repository setup

On top of what has been mentioned above, we are still investigating the right
approach for how the code for this effort should be created and organised. The
most likely approach will involve a monorepo (similar to
[`rnx-kit`](https://github.com/microsoft/rnx-kit)) where each module will be its
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's important for this to live in the react-native repo and not be another 3rd party project that people have to go looking around for to find modules. My position is that the DOM/Web APIs is the long-term future of the maintained RN JS APIs, and it's critical that they be hosted and maintained out of the main monorepo infra.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @necolas, it's great to know that we are very much aligned in exploring this direction 🤗 about this specific angle of where the code should live, while I agree that in the long-term all this xplat code should live together, I think that while this is still very much experimental it'd be better to keep it somewhere else, and then potentially migrate it in the react-native repo if/when we validate that it's viable.
I'm concerned that moving the code into rn now will create a lot of overhead in both maintenance and contribution speed, and during this first experimental phase we want to be able to move swiftly without worrying inadvertently breaking something else.
Let's chat again about this further down the line?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends how long "experimental" goes on for. We're currently greatly slowed by a lot of past forking and out-of-repo initiatives, and have long and complicated projects to defork those things. That needs to be avoided with new work.

We also have existing Web APIs that we've implemented and they're not separate repos or projects either. They'll be highly dependent on new internals like an event loop model and native extension points, so being out of core will make it harder for Web APIs with those needs to be tested and inform development.

Curious to hear @rubennorte's thoughts

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@necolas I think colocation is more critical for APIs that interact with internals from RN (like all the DOM traversal & layout APIs that interact with Fabric), but it might not be as important for standalone APIs like Battery, etc. that could be easily implemented as a userland module. I agree with the general direction of eventually having this provided by the framework out of the box, but I don't think it's that important at this point (and it'll help with iteration speed).

own dedicated package. For starters, we will suggest that implementations for
"core" supported platforms (i.e. Android, iOS, macOS, Windows) be present in the
one package — we won't have `/~/battery-status-android` and
`/~/battery-status-ios`, only `/~/battery-status`. However, it should still be
possible to have additional platform specific implementations, e.g.
`/~/battery-status-tvos`. These should also be treated as first class citizens
and be recognized by all tooling.

This should allow for different people to work on different modules at the same
time, while having a coherent infra to rely on for testing, publishing, etc.

### Discovery

The number of modules is high. Finding which modules provide which part of Web
APIs can be overwhelming for consumers. We need tools that can tell users which
dependencies they need to add. At minimum we should:
Copy link

@rubennorte rubennorte Sep 8, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my previous comment about autolinking and how it could completely solve the discovery problem :)


1. Implement a tool for detecting usage of web APIs
- The tool should be able to list used web APIs and flag uses that have not
been polyfilled.
- If possible, the tool should also recommend which dependencies to add
and/or automatically add it to `package.json`.
- **Note:** While we say "tool" here, it doesn't necessarily have to be a
standalone thing. A Babel plugin or similar would also fit. The less users
have to worry about it, the better.
Comment on lines +180 to +187
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that having auto-magic setup would be a great DX, and an essential part of opening up RN to developers coming from Web in the future.

2. Add new capabilities to [`@rnx-kit/align-deps`][]
- `align-deps` ensures that package versions are aligned and can help keeping
track of transitive dependencies.

## Drawbacks

- Existing React Native apps might need to be adapted this new paradigm.
- A lot of the tooling needed for this effort to succeed as detailed above needs
to be created.
- The number of Web APIs is very high. Implementing each and every one of them
for all the platforms will take a massive amount of time and funding —
realistically, we will select a subset of APIs to focus on, based on needs and
real usage data.

## Rationale, alternatives, and prior art

- A variation of the current proposal without polyfills was considered, but it
would require users to change web code to accommodate native. For instance,
`navigator.getBattery()` would have to be rewritten as
`require("@react-native-webapis/battery-status").getBattery()`.
tido64 marked this conversation as resolved.
Show resolved Hide resolved
- There are many polyfills out there, but they are mostly used to provide
functionalities that are only present in newer ES standards (e.g.
[`Object.assign`][], [`Object.is`][]). We have not found any that address the
scope defined here.
- [React Native Test App](https://github.com/microsoft/react-native-test-app)
will be used as the sandbox tool to build the modules against to reduce
friction and have "out of the box" the ability to test across all the target
platforms.
- This goal of web-like code working via React Native on multiple platforms is
shared with Meta's
[RFC: React DOM for Native](https://github.com/react-native-community/discussions-and-proposals/pull/496),
which should be considered complementary to the proposal presented here.

## Adoption strategy

We will be following the crawl-walk-run methodology:

- **Crawl stage:** Only one module will be implemented so that the tooling and
infrastructure can be built and flows can be set up. The infrastructure should
allow for multiple teams and developers to work discretely on the specific
modules that they are interested in, removing most of the friction to enable
contributors to just focus on producing code. We want to evaluate whether
things up to this point still make sense and adapt if otherwise.
kelset marked this conversation as resolved.
Show resolved Hide resolved

- **Walk stage:** One or more modules will be adopted as experimental within a
few selected production apps to validate the concept and implementation. This
implies that we have worked out what modules to prioritize by this stage.

- **Run stage:** By this stage, we aim to have a good level of confidence on the
viability of this proposal. Documentation on how to use this new set of
modules will be prepared to help developers leverage them to bring their web
code to native platforms.

## Unresolved questions

- Which parts of the Web API do we prioritize first?
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've been implementing MutationObserver and IntersectionObserver. Some Web APIs like those will come up against differences in the execution model of React Native vs Web, and we're working on an RFC to address that as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another note: to help us prioritize against our internal needs, we've been creating a tool to help us scrape a codebase to find usages of every WebAPI so that we can be data-driven #2621

- As proof-of-concept, we've implemented the
[`Battery Status API`](https://developer.mozilla.org/en-US/docs/Web/API/Battery_Status_API)
as it is small and self-contained.
- https://github.com/microsoft/rnx-kit/pull/2590
- Note that we're currently using
[`serializer.getModulesRunBeforeMainModule`][] until something better
exists. It currently requires that listed modules are explicitly imported
in the bundle itself, which we do not want. We need something akin to
Webpack's [entry points](https://webpack.js.org/concepts/entry-points/).
More details here: https://github.com/facebook/metro/issues/850.
kelset marked this conversation as resolved.
Show resolved Hide resolved

<!-- References -->

[`@rnx-kit/align-deps`]:
https://github.com/microsoft/rnx-kit/tree/main/packages/align-deps#readme
[`Object.assign`]: https://github.com/ljharb/object.assign/blob/main/polyfill.js
[`Object.is`]: https://github.com/es-shims/object-is/blob/main/polyfill.js
[`serializer.getModulesRunBeforeMainModule`]:
https://github.com/facebook/react-native/blob/0.72-stable/packages/metro-config/index.js#L49
[autolinking]:
https://github.com/react-native-community/cli/blob/main/docs/autolinking.md