Skip to content

Commit

Permalink
feat(cli, metro-config): environment variable support (#21983)
Browse files Browse the repository at this point in the history
# Why

- It's nice to be able to use uncommitted values in your app, based on
the environment. This feels very familiar to web developers.
- Values that are prefixed with `EXPO_PUBLIC_` will be inlined in the
bundle when bundling normally (e.g. not for Node.js).
- `.env` files are loaded into memory and applied to the process during
a run. This also means that they're available in `app.config.js`.
- During development-only, environment variables are exposed on the
`process.env` object (non-enumerable) to ensure they're available
between fast refresh updates.
<!--
Please describe the motivation for this PR, and link to relevant GitHub
issues, forums posts, or feature requests.
-->

# How

- Create new package `@expo/env` which is used to hydrate env variables
in a unified way. I plan to open another PR in `eas-cli` which uses this
package to fill in environment variables before uploading. NOTE:
environment variables that are defined in eas.json are not available in
Expo CLI when building locally, but are available in the cloud since
they'll be on the process, this means they effectively emulate
`.env.production`.
- Update templates to gitignore local env files.
- Add basic documentation to the versioned metro guide (more to come).


<!--
How did you build this feature or fix this bug and why?
-->

# Test Plan

- [ ] E2E rendering test
- [ ] E2E Node.js rendering test
- [x] Unit tests for serializer
- [x] Tests for env package

<!--
Please describe how you tested this change and how a reviewer could
reproduce your test, especially if this PR does not include automated
tests! If possible, please also provide terminal output and/or
screenshots demonstrating your test/reproduction.
-->

# Checklist

<!--
Please check the appropriate items below if they apply to your diff.
This is required for changes to Expo modules.
-->

- [ ] Documentation is up to date to reflect these changes (eg:
https://docs.expo.dev and README.md).
- [ ] Conforms with the [Documentation Writing Style
Guide](https://github.com/expo/expo/blob/main/guides/Expo%20Documentation%20Writing%20Style%20Guide.md)
- [ ] This diff will work correctly for `expo prebuild` & EAS Build (eg:
updated a module plugin).
  • Loading branch information
EvanBacon committed Apr 8, 2023
1 parent 2f0cf6a commit 6a750d0
Show file tree
Hide file tree
Showing 66 changed files with 1,517 additions and 45 deletions.
27 changes: 27 additions & 0 deletions docs/pages/versions/unversioned/config/metro.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,33 @@ title: metro.config.js

See more information about **metro.config.js** in the [customizing Metro guide](/guides/customizing-metro/).

## Envrionment variables

> For SDK 49 and above
Environment variables can be loaded using **.env** files. These files are loaded according to the [standard **.env** file resolution](https://github.com/bkeepers/dotenv/blob/c6e583a/README.md#what-other-env-files-can-i-use).

If you are migrating an older project to SDK 49 or above, then you should ignore local env files by adding the following to your **.gitignore**:

```sh .gitignore
# local env files
.env*.local
```

### Client envrionment variables

Envrionment variables prefixed with `EXPO_PUBLIC_` will be exposed to the app at build-time. For example, `EXPO_PUBLIC_API_KEY` will be available as `process.env.EXPO_PUBLIC_API_KEY`.

Envrionmnet variables will not be inlined in Node Modules.

For security purposes, client environment variables are inlined in the bundle, which means that `process.env` is not an iterable object, and you cannot dynamically access environment variables. Every variable must be explicitly defined. For example, `process.env.EXPO_PUBLIC_KEY` is valid, and `process.env['EXPO_PUBLIC_KEY']` is not.

- Client environment variables should not contain secrets as they will be viewable in plain-text format in the production binary.
- Use client environment variables for partially protected values, such as public API keys you don't want to commit to GitHub.
- Expo environment variables can be updated during `npx expo start` (without clearing the bundler cache). However, you'll need to modify and save an included source file to see the updates. You must also perform a client reload, as environment variables do not support Fast Refresh.

{/* TODO: Usage in EAS */}

## CSS

> Since SDK 49
Expand Down
3 changes: 2 additions & 1 deletion packages/@expo/cli/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,11 @@
- Add EXPO_ROUTER_TYPED_ROUTES flag to enable experimental support for type generation ([#21560](https://github.com/expo/expo/pull/21651) by [@marklawlor](https://github.com/marklawlor))
- Add inspector support for `Page.reload` CDP message. ([#21827](https://github.com/expo/expo/pull/21827) by [@byCedric](https://github.com/byCedric))
- Add Node.js rendering to Metro bundler and Node.js external imports. ([#21886](https://github.com/expo/expo/pull/21886) by [@EvanBacon](https://github.com/EvanBacon))
- Add support for inlining environment variables using the `EXPO_PUBLIC_` prefix. ([#21983](https://github.com/expo/expo/pull/21983) by [@EvanBacon](https://github.com/EvanBacon))
- Add support for loading environment variables from `.env` files. ([#21983](https://github.com/expo/expo/pull/21983) by [@EvanBacon](https://github.com/EvanBacon))
- Add support for emitting static CSS files when exporting web projects with `expo export`. ([#21941](https://github.com/expo/expo/pull/21941) by [@EvanBacon](https://github.com/EvanBacon))
- Remove legacy manifest signing and fall back to unsigned when insufficient account permission to sign. ([#21989](https://github.com/expo/expo/pull/21989) by [@wschurman](https://github.com/wschurman))


### 🐛 Bug fixes

- Respond to `Debugger.getScriptSource` CDP messages when using lan or tunnel. ([#21825](https://github.com/expo/expo/pull/21825) by [@byCedric](https://github.com/byCedric))
Expand Down
2 changes: 2 additions & 0 deletions packages/@expo/cli/e2e/fixtures/with-router/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
EXPO_PUBLIC_DEMO_KEY=DEMO_KEY
INVALID_DEMO_KEY=INVALID
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
EXPO_PUBLIC_DEMO_KEY=DEMO_KEY_PROD
1 change: 1 addition & 0 deletions packages/@expo/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
"@babel/runtime": "^7.20.0",
"@expo/code-signing-certificates": "0.0.5",
"@expo/config": "~8.0.0",
"@expo/env": "~0.0.0",
"@expo/config-plugins": "~6.0.0",
"@expo/dev-server": "0.2.2",
"@expo/devcert": "^1.0.0",
Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/config/configAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ export function logConfig(config: ExpoConfig | ProjectConfig) {

export async function configAsync(projectRoot: string, options: Options) {
setNodeEnv('development');
require('@expo/env').load(projectRoot);

if (options.type) {
assert.match(options.type, /^(public|prebuild|introspect)$/);
Expand Down
2 changes: 2 additions & 0 deletions packages/@expo/cli/src/customize/customizeAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ export async function customizeAsync(files: string[], options: Options, extras:
// This enables users to run `npx expo customize` from a subdirectory of the project.
const projectRoot = findUpProjectRootOrAssert(process.cwd());

require('@expo/env').load(projectRoot);

// Get the static path (defaults to 'web/')
// Doesn't matter if expo is installed or which mode is used.
const { exp } = getConfig(projectRoot, {
Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/export/embed/exportEmbedAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { Options } from './resolveOptions';

export async function exportEmbedAsync(projectRoot: string, options: Options) {
setNodeEnv(options.dev ? 'development' : 'production');
require('@expo/env').load(projectRoot);

const { config } = await loadMetroConfigAsync(projectRoot, {
maxWorkers: options.maxWorkers,
Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/export/exportApp.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ export async function exportAppAsync(
}: Pick<Options, 'dumpAssetmap' | 'dumpSourcemap' | 'dev' | 'clear' | 'outputDir' | 'platforms'>
): Promise<void> {
setNodeEnv(dev ? 'development' : 'production');
require('@expo/env').load(projectRoot);

const exp = await getPublicExpoManifestAsync(projectRoot);

Expand Down
4 changes: 4 additions & 0 deletions packages/@expo/cli/src/export/web/exportWebAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,16 @@ import { WebSupportProjectPrerequisite } from '../../start/doctor/web/WebSupport
import { getPlatformBundlers } from '../../start/server/platformBundlers';
import { WebpackBundlerDevServer } from '../../start/server/webpack/WebpackBundlerDevServer';
import { CommandError } from '../../utils/errors';
import { setNodeEnv } from '../../utils/nodeEnv';
import { Options } from './resolveOptions';

export async function exportWebAsync(projectRoot: string, options: Options) {
// Ensure webpack is available
await new WebSupportProjectPrerequisite(projectRoot).assertAsync();

setNodeEnv(options.dev ? 'development' : 'production');
require('@expo/env').load(projectRoot);

const { exp } = getConfig(projectRoot);
const platformBundlers = getPlatformBundlers(exp);
// Create a bundler interface
Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/install/installAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ export async function installAsync(
// Locate the project root based on the process current working directory.
// This enables users to run `npx expo install` from a subdirectory of the project.
const projectRoot = options.projectRoot ?? findUpProjectRootOrAssert(process.cwd());
require('@expo/env').load(projectRoot);

// Resolve the package manager used by the project, or based on the provided arguments.
const packageManager = PackageManager.createForProject(projectRoot, {
Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/prebuild/prebuildAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ export async function prebuildAsync(
}
): Promise<PrebuildResults | null> {
setNodeEnv('development');
require('@expo/env').load(projectRoot);

if (options.clean) {
const { maybeBailOnGitStatusAsync } = await import('../utils/git');
Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/run/__tests__/startBundler-test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ jest.mock('../../start/server/DevServerManager', () => ({
startAsync: jest.fn(),
getDefaultDevServer: jest.fn(),
bootstrapTypeScriptAsync: jest.fn(),
watchEnvironmentVariables: jest.fn(),
})),
}));

Expand Down
5 changes: 3 additions & 2 deletions packages/@expo/cli/src/run/android/runAndroidAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ import { Options, ResolvedOptions, resolveOptionsAsync } from './resolveOptions'
const debug = require('debug')('expo:run:android');

export async function runAndroidAsync(projectRoot: string, { install, ...options }: Options) {
// TODO: Add support for setting as production.
setNodeEnv('development');
// NOTE: This is a guess, the developer can overwrite with `NODE_ENV`.
setNodeEnv(options.variant === 'release' ? 'production' : 'development');
require('@expo/env').load(projectRoot);

await ensureNativeProjectAsync(projectRoot, { platform: 'android', install });

Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/run/ios/runIosAsync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import { resolveOptionsAsync } from './options/resolveOptions';

export async function runIosAsync(projectRoot: string, options: Options) {
setNodeEnv(options.configuration === 'Release' ? 'production' : 'development');
require('@expo/env').load(projectRoot);

assertPlatform();

Expand Down
1 change: 1 addition & 0 deletions packages/@expo/cli/src/run/startBundler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ export async function startBundlerAsync(
}

if (!options.headless) {
await devServerManager.watchEnvironmentVariables();
await devServerManager.bootstrapTypeScriptAsync();
}

Expand Down
4 changes: 4 additions & 0 deletions packages/@expo/cli/src/start/server/BundlerDevServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,10 @@ export abstract class BundlerDevServer {
// noop -- We've only implemented this functionality in Metro.
}

public async watchEnvironmentVariables(): Promise<void> {
// noop -- We've only implemented this functionality in Metro.
}

/**
* Creates a mock server representation that can be used to estimate URLs for a server started in another process.
* This is used for the run commands where you can reuse the server from a previous run.
Expand Down
4 changes: 4 additions & 0 deletions packages/@expo/cli/src/start/server/DevServerManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,10 @@ export class DevServerManager {
}
}

async watchEnvironmentVariables() {
await devServers.find((server) => server.name === 'metro')?.watchEnvironmentVariables();
}

/** Stop all servers including ADB. */
async stopAsync(): Promise<void> {
await Promise.allSettled([
Expand Down
34 changes: 34 additions & 0 deletions packages/@expo/cli/src/start/server/metro/MetroBundlerDevServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@
*/
import { getConfig } from '@expo/config';
import { prependMiddleware } from '@expo/dev-server';
import * as runtimeEnv from '@expo/env';
import assert from 'assert';
import chalk from 'chalk';
import path from 'path';

import { Log } from '../../../log';
import getDevClientProperties from '../../../utils/analytics/getDevClientProperties';
Expand All @@ -29,6 +31,7 @@ import { ServerNext, ServerRequest, ServerResponse } from '../middleware/server.
import { typescriptTypeGeneration } from '../type-generation';
import { instantiateMetroAsync } from './instantiateMetro';
import { metroWatchTypeScriptFiles } from './metroWatchTypeScriptFiles';
import { observeFileChanges } from './waitForMetroToObserveTypeScriptFile';

const debug = require('debug')('expo:start:server:metro') as typeof console.log;

Expand Down Expand Up @@ -90,6 +93,37 @@ export class MetroBundlerDevServer extends BundlerDevServer {
return await load(location);
}

async watchEnvironmentVariables() {
if (!this.instance) {
throw new Error(
'Cannot observe environment variable changes without a running Metro instance.'
);
}
if (!this.metro) {
// This can happen when the run command is used and the server is already running in another
// process.
debug('Skipping Environment Variable observation because Metro is not running (headless).');
return;
}

const envFiles = runtimeEnv
.getFiles(process.env.NODE_ENV)
.map((fileName) => path.join(this.projectRoot, fileName));

observeFileChanges(
{
metro: this.metro,
server: this.instance.server,
},
envFiles,
() => {
debug('Reloading environment variables...');
// Force reload the environment variables.
runtimeEnv.load(this.projectRoot, { force: true });
}
);
}

protected async startImplementationAsync(
options: BundlerStartOptions
): Promise<DevServerInstance> {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
import path from 'path';

import type { ServerLike } from '../BundlerDevServer';

const debug = require('debug')('expo:start:server:metro:waitForTypescript') as typeof console.log;

/**
* Use the native file watcher / Metro ruleset to detect if a
* TypeScript file is added to the project during development.
*/
export function waitForMetroToObserveTypeScriptFile(
projectRoot: string,
runner: {
metro: import('metro').Server;
server: ServerLike;
},
callback: () => Promise<void>
): () => void {
const watcher = runner.metro.getBundler().getBundler().getWatcher();

const tsconfigPath = path.join(projectRoot, 'tsconfig.json');

const listener = ({
eventsQueue,
}: {
eventsQueue: {
filePath: string;
metadata?: {
type: 'f' | 'd' | 'l'; // Regular file / Directory / Symlink
} | null;
type: string;
}[];
}) => {
for (const event of eventsQueue) {
if (
event.type === 'add' &&
event.metadata?.type !== 'd' &&
// We need to ignore node_modules because Metro will add all of the files in node_modules to the watcher.
!/node_modules/.test(event.filePath)
) {
const { filePath } = event;
// Is TypeScript?
if (
// If the user adds a TypeScript file to the observable files in their project.
/\.tsx?$/.test(filePath) ||
// Or if the user adds a tsconfig.json file to the project root.
filePath === tsconfigPath
) {
debug('Detected TypeScript file added to the project: ', filePath);
callback();
off();
return;
}
}
}
};

debug('Waiting for TypeScript files to be added to the project...');
watcher.addListener('change', listener);

const off = () => {
watcher.removeListener('change', listener);
};

runner.server.addListener?.('close', off);
return off;
}

export function observeFileChanges(
runner: {
metro: import('metro').Server;
server: ServerLike;
},
files: string[],
callback: () => void | Promise<void>
): () => void {
const watcher = runner.metro.getBundler().getBundler().getWatcher();

const listener = ({
eventsQueue,
}: {
eventsQueue: {
filePath: string;
metadata?: {
type: 'f' | 'd' | 'l'; // Regular file / Directory / Symlink
} | null;
type: string;
}[];
}) => {
for (const event of eventsQueue) {
if (
// event.type === 'add' &&
event.metadata?.type !== 'd' &&
// We need to ignore node_modules because Metro will add all of the files in node_modules to the watcher.
!/node_modules/.test(event.filePath)
) {
const { filePath } = event;
// Is TypeScript?
if (files.includes(filePath)) {
debug('Observed change:', filePath);
callback();
return;
}
}
}
};

debug('Watching file changes:', files);
watcher.addListener('change', listener);

const off = () => {
watcher.removeListener('change', listener);
};

runner.server.addListener?.('close', off);
return off;
}
Original file line number Diff line number Diff line change
Expand Up @@ -313,9 +313,11 @@ export async function withMetroMultiPlatformAsync(
platformBundlers: PlatformBundlers;
}
) {
// Auto pick App entry: this is injected with Babel.
// Auto pick App entry: this is injected with a custom serializer.
process.env.EXPO_ROUTER_APP_ROOT = getAppRouterRelativeEntryPath(projectRoot);
process.env.EXPO_PROJECT_ROOT = process.env.EXPO_PROJECT_ROOT ?? projectRoot;

// Required for @expo/metro-runtime to format paths in the web LogBox.
process.env.EXPO_PUBLIC_PROJECT_ROOT = process.env.EXPO_PUBLIC_PROJECT_ROOT ?? projectRoot;

if (env.EXPO_USE_STATIC) {
// Enable static rendering in runtime space.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
/**
* Copyright © 2022 650 Industries.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
import fs from 'fs';
import path from 'path';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,7 @@ export class WebpackBundlerDevServer extends BundlerDevServer {
https: options.https,
};
setNodeEnv(env.mode ?? 'development');
require('@expo/env').load(env.projectRoot);
// Check if the project has a webpack.config.js in the root.
const projectWebpackConfig = this.getProjectConfigFilePath();
let config: WebpackConfiguration;
Expand Down
Loading

0 comments on commit 6a750d0

Please sign in to comment.