[expo-updates] Add fingerprintExperimental runtime version policy #24126

mccraveiro · 2023-08-27T04:03:11Z

Why

Manually handling runtimeVersions is a big pain for projects that uses native modules. This PR adds a new runtime policy that handles this automatically.

See:
https://discord.com/channels/695411232856997968/1142023605908156476/1142023605908156476

How

Using expo-fingerprint to auto-generate a hash for the project.

Test Plan

I made the changes locally on a working project and the build was successfully with expo prebuild. I checked the native files and the runtime version was set to the project hash.

Checklist

Documentation is up to date to reflect these changes (eg: https://docs.expo.dev and README.md).
Conforms with the Documentation Writing Style Guide
This diff will work correctly for npx expo prebuild & EAS Build (eg: updated a module plugin).

ItsWendell · 2023-08-27T06:27:50Z

Nice this would definitely be cool to have! Here are some interesting, relevant conversations around @expo/fingerprint:

#24069

It still has some edge cases (e.g. monorepos), but it's getting there! In a script that I wrote, I do something similar as you do here but then store the hash in a json file which I then exclude from the fingerprint, and import in my app.config.ts file.

The beauty of using @expo/fingerprint is that it outputs its sources and the individual hash per source, so you could use this to 'tweak' the hash to be optimal for runtime versions.

E.g. do you want your eas.json file or your package.json scripts to be included in the hash for your runtime version, I think what is included in your fingerprint for runtime versions is something to think about, document and discuss, and to allow possibly for customizations where needed.

quinlanj

This is a good first pass! The fingerprint policy is also on our roadmap. Reading the discord and PR thread, here is what the fingerprint runtime policy could look like as it becomes more mature:

Features

Users should be able to audit their fingerprint and understand it’s structure. It'll be helpful for folks to see the input sources that go into generating the fingerprint. Possible solution: write fingerprint.json to project directory every time we generate runtime
Ability to customize input sources. @ItsWendell does this in a script, but when we expose a fingerprint policy, we should make it easier for folks to achieve the same thing. Possible solution: expose an .ignore file for fingerprint

Stability

A couple concerns about releasing it to production today.

fingerprint package is still quite new. We still need to dogfood the edge cases and ensure we can produce stable hashes under different conditions (ie) monorepos, different OS’es, etc
An option is to tell users about the experimental nature of the fingerprint policy in the docs and in a Log.warn in eas-cli.

packages/@expo/config-plugins/src/utils/Updates.ts

Kudo · 2023-08-31T00:39:10Z

Possible solution: expose an .ignore file for fingerprint

that's great idea. i will take this one for fingerprint.

no async getRuntimeVersionNullable

since this pr changes the getRuntimeVersionNullable interface, how about just introducing new getRuntimeVersionNullableAsync and deprecating the original function? maybe it's safer than just changing the original interface for eas-cli integration.

mccraveiro · 2023-08-31T01:31:33Z

Ability to customize input sources. @ItsWendell does this in a script, but when we expose a fingerprint policy, we should make it easier for folks to achieve the same thing. Possible solution: expose an .ignore file for fingerprint

@quinlanj I'm not sure I get the use case where someone would customize the files that goes into the fingerprint. On @ItsWendell's script this seems to be the case because of an underlining bug in expo-fingerprint. Not sure we should plan around that.

An option is to tell users about the experimental nature of the fingerprint policy in the docs and in a Log.warn in eas-cli.

@quinlanj I agree 100% with this. I'll update the code and documentation.

since this pr changes the getRuntimeVersionNullable interface, how about just introducing new getRuntimeVersionNullableAsync and deprecating the original function? maybe it's safer than just changing the original interface for eas-cli integration.

@Kudo I'm not sure this achieves the desired goal. As stated on Expo's Config Plugins page: "Plugins are synchronous functions". Therefore, creating a getRuntimeVersionNullableAsync wouldn't be enough. For this to work truly asynchronous we would need to rewrite how config plugins work.

Kudo · 2023-08-31T01:56:23Z

@Kudo I'm not sure this achieves the desired goal. As stated on Expo's Config Plugins page: "Plugins are synchronous functions". Therefore, creating a getRuntimeVersionNullableAsync wouldn't be enough. For this to work truly asynchronous we would need to rewrite how config plugins work.

plugins are synchronous but mods could be asynchronous. an example is like this one. anyhow, i agree that introducing getRuntimeVersionNullableAsync requires a lot of changes since current update related config-plugins code is not async. but after the change, it could open more possibilities. (though just to discuss something we could enhance internally in the future)

mccraveiro · 2023-08-31T02:08:45Z

An option is to tell users about the experimental nature of the fingerprint policy in the docs and in a Log.warn in eas-cli.

@quinlanj It seems that we would also need a Log.warn in expo-cli. Any tips on how to handle this and reduce code duplication?

mccraveiro · 2023-08-31T02:54:49Z

@Kudo @quinlanj I updated the code to work asynchronous and It was easier than expected. Not sure I'm missing something.

quinlanj

Couple nits, but otherwise lgtm 🚀 Let's also wait for @douglowder or @wschurman to take a look as well.

In response to the other messages on this thread:

I'm not sure I get the use case where someone would customize the files that goes into the fingerprint. On @ItsWendell's script this seems to be the case because of an underlining bug in expo-fingerprint. Not sure we should plan around that.

Here are my hypothesis on why people would want to customize:

Fingerprint returns different hash, but user wants equivalent runtime: Developer has extra information that fingerprint doesnt. For example, if they made a change in app.json they know doesn't affect native compatibility.
Fingerprint returns same hash, but user wants different runtime: Developer has made JS/TS code changes that is compatible with the native code but is incompatible from a product standpoint, so they want to bump the runtime version.

These are just my hypothesis, so I'm curious if any of these cases resonate with you.

It seems that we would also need a Log.warn in expo-cli. Any tips on how to handle this and reduce code duplication?

I saw you added console.warn. This is sufficient and will print when the cli is run, so we can skip adding Log.warn

packages/@expo/config-plugins/src/android/Updates.ts

docs/pages/eas-update/runtime-versions.mdx

mccraveiro · 2023-09-03T01:56:05Z

Here are my hypothesis on why people would want to customize:

Fingerprint returns different hash, but user wants equivalent runtime: Developer has extra information that fingerprint doesnt. For example, if they made a change in app.json they know doesn't affect native compatibility.

Fingerprint returns same hash, but user wants different runtime: Developer has made JS/TS code changes that is compatible with the native code but is incompatible from a product standpoint, so they want to bump the runtime version.

These are just my hypothesis, so I'm curious if any of these cases resonate with you.

@quinlanj, you bring up interesting use cases. Still, I think the existing custom runtime version covers those bases. The fingerprint is more for people who want a 'set-it-and-forget-it' option for releases.

Kudo

since this pr introduced breaking changes. we should also update eas-cli and eas-build to migrate for the new getRuntimeVersionAsync and getRuntimeVersionNullableAsync. that would be something we need to follow up after the pr landed.

docs/pages/eas-update/runtime-versions.mdx

packages/@expo/config-plugins/CHANGELOG.md

packages/@expo/config-plugins/src/android/Updates.ts

packages/@expo/config-plugins/src/utils/Updates.ts

Co-authored-by: Kudo Chien <ckchien@gmail.com>

douglowder · 2023-09-06T20:49:28Z

Generally looks good. Will take a look to see what is causing Updates E2E to fail.

quinlanj

@mccraveiro I spoke to various members of our team and we'd like to rename the policy name to fingerprintExperimental. Many folks don't read the docs or console outputs, so this is an additional way for us to communicate that this feature is experimental.

Thanks again for navigating the discord threads, putting together this PR and implementing all our feedback. Really appreciate it!

mccraveiro · 2023-09-07T01:51:07Z

@quinlanj That makes sense. I'll update the PR.

Kudo

it's almost there! sorry that i just found one problem from a last-minute review. after that it's perfect for me.

the ci updates e2e tests failures on eas build are expected for external contributors.

packages/@expo/cli/src/start/server/middleware/ExpoGoManifestHandlerMiddleware.ts

…andlerMiddleware.ts Co-authored-by: Kudo Chien <ckchien@gmail.com>

# Why add `.fingerprintignore` support which is commented from #24126 (review) # How - add `options.ignorePaths` that support both dirs/files and try to support patterns more like .gitignore - read `{projectRoot}/.fingerprintignore` as additional `options.ignorePaths` usage example: add `{projectRoot/.fingerprintignore`. the contents could be as following example ``` # Comment prefixed with # is supported # ignores whole ios directory ios/**/* # but not ios/Podfile !ios/Podfile ```

packages/@expo/config-plugins/package.json

fingerprint dependency needs to be 0.2.0 to be compatible with the monorepo

douglowder

Fixed fingerprint dependency, resolved CHANGELOG conflict, and ran updates E2E tests manually to verify that they pass. Looks good. Will allow CI to run before merging. @quinlanj

mccraveiro · 2023-09-11T19:51:01Z

Thanks @douglowder!

…4126) # Why Manually handling runtimeVersions is a big pain for projects that uses native modules. This PR adds a new runtime policy that handles this automatically. See: https://discord.com/channels/695411232856997968/1142023605908156476/1142023605908156476 # How Using [expo-fingerprint](https://github.com/expo/expo/tree/main/packages/@expo/fingerprint#readme) to auto-generate a hash for the project. # Test Plan I made the changes locally on a working project and the build was successfully with expo prebuild. I checked the native files and the runtime version was set to the project hash. # Checklist - [x] Documentation is up to date to reflect these changes (eg: https://docs.expo.dev and README.md). - [x] Conforms with the [Documentation Writing Style Guide](https://github.com/expo/expo/blob/main/guides/Expo%20Documentation%20Writing%20Style%20Guide.md) - [x] This diff will work correctly for `npx expo prebuild` & EAS Build (eg: updated a module plugin). --------- Co-authored-by: Expo Bot <34669131+expo-bot@users.noreply.github.com> Co-authored-by: Aman Mittal <amandeepmittal@live.com> Co-authored-by: Kudo Chien <ckchien@gmail.com> Co-authored-by: Douglas Lowder <douglowder@mac.com>

a-eid · 2024-01-19T10:26:21Z

after upgrading to SDK 50 I get the following error when I run my dev client ( SDK 50 ).

CommandError: Must specify --private-key-path argument to sign development manifest for requested code signing key

wschurman · 2024-01-19T17:21:39Z

@a-eid - Please file a new issue with more information including a repro, the commands you are running, etc. https://github.com/expo/expo/issues/new/choose

mccraveiro requested review from Simek, amandeepmittal, EvanBacon, douglowder, brentvatne and byCedric as code owners August 27, 2023 04:03

expo-bot added the bot: suggestions ExpoBot has some suggestions label Aug 27, 2023

brentvatne requested review from Kudo, wschurman and quinlanj and removed request for Simek, byCedric, EvanBacon and amandeepmittal August 29, 2023 22:22

quinlanj reviewed Aug 30, 2023

View reviewed changes

packages/@expo/config-plugins/src/utils/Updates.ts Outdated Show resolved Hide resolved

quinlanj self-requested a review August 30, 2023 21:21

This comment was marked as resolved.

Sign in to view

quinlanj approved these changes Aug 31, 2023

View reviewed changes

packages/@expo/config-plugins/src/android/Updates.ts Outdated Show resolved Hide resolved

amandeepmittal reviewed Sep 1, 2023

View reviewed changes

docs/pages/eas-update/runtime-versions.mdx Outdated Show resolved Hide resolved

docs/pages/eas-update/runtime-versions.mdx Outdated Show resolved Hide resolved

docs/pages/eas-update/runtime-versions.mdx Outdated Show resolved Hide resolved

mccraveiro requested a review from amandeepmittal September 3, 2023 01:43

mccraveiro requested a review from quinlanj September 3, 2023 01:56

Kudo mentioned this pull request Sep 4, 2023

[fingerprint] Add .fingerprintignore support #24265

Merged

3 tasks

[expo-updates] Add fingerprint runtime version policy

5ce9bae

Kudo reviewed Sep 6, 2023

View reviewed changes

mccraveiro and others added 2 commits September 6, 2023 04:13

Update docs/pages/eas-update/runtime-versions.mdx

3b713d1

Co-authored-by: Kudo Chien <ckchien@gmail.com>

Update packages/@expo/config-plugins/src/android/Updates.ts

3bb9214

Co-authored-by: Kudo Chien <ckchien@gmail.com>

mccraveiro added 3 commits September 6, 2023 18:34

Fix types

ac5d101

Update changelog with breaking changes

11bd060

Remove unused withRuntimeVersionAsync mod

ae05a80

mccraveiro requested a review from Kudo September 6, 2023 21:39

wschurman removed their request for review September 6, 2023 23:32

quinlanj reviewed Sep 7, 2023

View reviewed changes

Rename runtime policy to fingerprintExperimental

4737378

mccraveiro changed the title ~~[expo-updates] Add fingerprint runtime version policy~~ [expo-updates] Add fingerprintExperimental runtime version policy Sep 7, 2023

Fix failing test

f84b3e0

Kudo reviewed Sep 7, 2023

View reviewed changes

packages/@expo/cli/src/start/server/middleware/ExpoGoManifestHandlerMiddleware.ts Outdated Show resolved Hide resolved

Update packages/@expo/cli/src/start/server/middleware/ExpoGoManifestH…

a15881b

…andlerMiddleware.ts Co-authored-by: Kudo Chien <ckchien@gmail.com>

Kudo approved these changes Sep 7, 2023

View reviewed changes

quinlanj approved these changes Sep 7, 2023

View reviewed changes

Merge branch 'main' into main

1e707d8

douglowder reviewed Sep 11, 2023

View reviewed changes

packages/@expo/config-plugins/package.json Outdated Show resolved Hide resolved

Bump fingerprint dependency

4fa4e94

fingerprint dependency needs to be 0.2.0 to be compatible with the monorepo

douglowder self-requested a review September 11, 2023 19:19

douglowder approved these changes Sep 11, 2023

View reviewed changes

douglowder merged commit f0d67e1 into expo:main Sep 11, 2023
11 of 14 checks passed

wschurman mentioned this pull request Feb 1, 2024

[config-plugins] Fix fingerprint runtime version policy #26839

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[expo-updates] Add fingerprintExperimental runtime version policy #24126

[expo-updates] Add fingerprintExperimental runtime version policy #24126

mccraveiro commented Aug 27, 2023 •

edited

ItsWendell commented Aug 27, 2023

quinlanj left a comment •

edited

Kudo commented Aug 31, 2023

mccraveiro commented Aug 31, 2023

Kudo commented Aug 31, 2023

This comment was marked as resolved.

mccraveiro commented Aug 31, 2023

mccraveiro commented Aug 31, 2023

quinlanj left a comment

mccraveiro commented Sep 3, 2023

Kudo left a comment

douglowder commented Sep 6, 2023

quinlanj left a comment

mccraveiro commented Sep 7, 2023

Kudo left a comment

douglowder left a comment

mccraveiro commented Sep 11, 2023

a-eid commented Jan 19, 2024

wschurman commented Jan 19, 2024 •

edited

[expo-updates] Add fingerprintExperimental runtime version policy #24126

[expo-updates] Add fingerprintExperimental runtime version policy #24126

Conversation

mccraveiro commented Aug 27, 2023 • edited

Why

How

Test Plan

Checklist

ItsWendell commented Aug 27, 2023

quinlanj left a comment • edited

Choose a reason for hiding this comment

Features

Stability

Kudo commented Aug 31, 2023

mccraveiro commented Aug 31, 2023

Kudo commented Aug 31, 2023

This comment was marked as resolved.

mccraveiro commented Aug 31, 2023

mccraveiro commented Aug 31, 2023

quinlanj left a comment

Choose a reason for hiding this comment

mccraveiro commented Sep 3, 2023

Kudo left a comment

Choose a reason for hiding this comment

douglowder commented Sep 6, 2023

quinlanj left a comment

Choose a reason for hiding this comment

mccraveiro commented Sep 7, 2023

Kudo left a comment

Choose a reason for hiding this comment

douglowder left a comment

Choose a reason for hiding this comment

mccraveiro commented Sep 11, 2023

a-eid commented Jan 19, 2024

wschurman commented Jan 19, 2024 • edited

mccraveiro commented Aug 27, 2023 •

edited

quinlanj left a comment •

edited

wschurman commented Jan 19, 2024 •

edited