Skip to content

Improving the developer experience by leveraging a dependency graph and abstracting away native implementation details #318

@pepicrft

Description

@pepicrft

Introduction

At Shopify, React Native is our default technology for building mobile apps. We use the official React Native CLI in combination with some internal tooling to interact with their projects. The setup has been working fine, but the experience we’d like developers to have is not quite there yet. We think there’s some room for improvement and that’s what I’d like to discuss this issue.

Before jumping into the details, I’d like to first share what I mean when I say there’s room for improvement. In the following section, I’ll be putting myself in the shoes of our developers and share some of the inconveniences that developers stumble upon when building apps for iOS and Android using React Native and its official CLI.

Even though they are not be related to each other, the solution that I’m proposing further down will help mitigate all of them. Note that they are numerated so that I can reference them from the proposed solution.

Motivation

  • 1. Migrations remain cumbersome: It’s a manual process that some developers follow in an “I-don’t-know-what-I’m-doing” mode: I’ve been told to put this line on the Podfile, add this other line on the AppDelegate, add this line in the Gradle file, and things should work. We’d like this process to be seamless so that developers feel encouraged to be in the latest version and therefore benefit from the constant improvements that are landing on the framework. Fully-automating the process is a dream, but I think we can take steps to get closer to that.
  • 2. Auto-linking and CocoaPods: Many developers are not aware that auto-linking is necessary for their apps to compile. While auto-linking happens implicitly at build time, on iOS it requires an extra pod install command and a properly-configured Podfile to integrate the dependencies in the Xcode project. All it takes to go from “every works” state to “my app doesn’t compile” is forgetting about running pod install, changing something in the Xcode project, or having an invalid configuration in the Podfile. Internet is full of suggestions along the lines of “add this build setting” and it’ll work, or add this Ruby code to your Podfile and I assure you it’ll work. Moreover, there are also chances that someone uses the global Pod as opposed to the version specified in the Gemfile, leading to inconsistent results across environments. Bear in mind that I’m not blaming CocoaPods here. It does a great job, but the setup RN CLI + CocoaPods + Ruby environment will make providing a great developer experience challenging.
  • 3. Binary caching across environments: More than an inconvenience, this is an improvement that I think I'd make a huge difference in regards to productivity. Assuming most of the time developers work on the Javascript layer, there’s no need to be compiling the native code when someone has done that already. That’s what Gradle and Xcode’s build system does locally by reusing past builds, but what if those artifacts can be shared with others? What if I’m a new developer in the team and I run react-native run ios and the time it takes to launch the app is the result of the time it takes to pull the binary and have it running on the simulator and pointing to my local Metro server?

Details

I think the React Native CLI could have an in-memory graph representation of the React Native packages that contain native code, and make compilation artifacts such as Gradle files or Xcode projects an implementation detail that is generated when needed and therefore doesn’t need to be part of the repository.

Projects would have a manifest file that describes the project. We can reuse the existing manifest for that or create a new one. The structure could be something along the lines of:

# Example of an app manifest

name: MyApp
targets:
  - name: MyApp
    product: app
    platform: iOS
    sources: iOS/**/*.{h.m}
    minimum_deployment_target: 13.4
    resources: Resources/iOS/**/*
    build_identifier: com.shopify.MyApp
    dependencies:
      - ReactNativeStorage
  - name: MyApp
    product: app
    platform: Android
    sources: Android/**/*.{kotlin}
    package_name: com.shopify.MyApp
      - ReactNativeStorage

# Example of a library manifest

name: ReactNativeStorage
targets:
  - name: ReactNativeStorage
    product: module # (alternatively library/package)
    platform: iOS
    sources: iOS/**/*.{h.m}
    minimum_deployment_target: 13.4
    resources: Resources/iOS/**/*
    build_identifier: com.shopify.MyApp
  - name: ReactNativeStorage
    product: module
    platform: Android
    sources: Android/**/*.{kotlin}
    package_name: com.shopify.MyApp

NPM is the source of truth of the dependency graph. However, manifests would explicitly define the native dependencies they’d like to have. Projects could have multiple targets (e.g. a target per supported platform), and the format of the targets would be different depending on the platform they are targeting. For example, while iOS identifies buildable units (targets) with what they call “build identifier”, Android uses package names.

Projects would not contain .podspec, Xcode Projects, nor Gradle files. They are generated when needed under a .react-native directory that is .gitignored. For example, if I run react-native run ios, the React Native CLI would know that we need an Xcode project for that, it’d generate it under that directory, compile it, and run it on the simulator. On iOS, the generated Xcode project would be a representation of the dependency graph, containing all the targets, linking settings, and phases. Therefore, it removes the need for having to install CocoaPods and run extra pod install steps. On Android, since the resolution of the dependency graph happens when Gradle launches the project, Gradle could proxy with the CLI to get the graph and translate that to build steps. All that logic could be implemented in Kotlin. By doing this we’d be tackling point 2. from above, Auto-linking and CocoaPods, but we’d need to do a fair amount of work to re-implement all the linking logic that CocoaPods has developed and improved over the years. Luckily, that’s something I’ve done recently in one of my side projects, Tuist, and I have a lot of contexts to translate that to the Javascript domain.

By doing this, we are also easing the migrations (point 1 from above) since developers no longer have to know what changes need to be made to their Xcode projects, Podfiles, or Build.gradle files. The React Native CLI would combine the dependency graph with the React Native version the project is using, and generate the right project for it. For instance, if the version of React Native comes with Hermes support for iOS, the generated Xcode project would contain the macros, and the right linking to enable Hermes. Developers don’t have to do anything.

Moreover, having a dependency graph allows for fingerprinting. Each target would have a fingerprint that is the result of fingerprinting the target’s settings, files, and the fingerprint of its dependencies. Why is that useful? If an app has a fingerprint, we can extend the react-native run ios to look up the binary in a remote cache using the fingerprint. For example, building upon the above example, we’d get the following fingerprint for MyApp, 1234 , so if the MyApp-1234.tar.gz exists in a remote cache, I’d take that one instead of going through the normal flow of generating, building and launching the project. Teams could set up a remote CI pipeline with the sole goal of warming the cache. This would tackle the point 3, Binary caching across environments.

Other benefits

I believe going down this path has some other benefits that are worth considering when evaluating it.
First, we’d ease the creation of libraries for React Native. A developer would only need to add a manifest file, a package.json, and the source code (JS/Typescript, Swift, Kotlin…). No Xcode projects, no Podspecs, Build.gradle files. We could add a new command, react-native edit that opens either Xcode or Android studio to edit the project.

Moreover, we could add graph validation. This is something that Gradle and CocoaPods do, but the errors that they throw, although accurate, are not actionable for React Native developers. We could make sure that the errors are clear and actionable and don’t let developers compile their projects if we know 100% that they won’t compile.

Another interesting idea that we recently explored in Tuist is the idea of focusing on projects (I believe Facebook has something similar). This is something that’d be possible if we have a graph. The idea being is that as a developer I can get a native project with a focus on one of the targets of the dependency tree. For example, let’s say that I’m having issues with one of the dependencies that I’m using, ReactNativeWhatever. I could run react-native focus ReactNativeWhatever and I’d get a project generated on-the-fly that would open on my editor and where I can build and test the native code easily.

Discussion points

  • Is this a potential direction the React Native CLI could take?
  • Is it worth re-implementing part of the work that CocoaPods has already done in the past?
  • How does this translate to other platforms (Windows, Mac)?


Metadata

Metadata

Assignees

No one assigned

    Labels

    🗣 DiscussionThis label identifies an ongoing discussion on a subject

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions