Validating environment variables with Astro Server-side Rendering

[intagaming]

Note: The following is kind of like a recipe page awaiting its debut in the t3-env docs. Please consider this a proposal before I (or someone else) do so.

When Astro is built with the default Static Site Generation (SSG), the validation will happen on each static route generation at the place of env usage. Technically this is "Validation on build".

However, with Astro Server-side Rendering (SSR), validation takes a different "route" (no pun intended). If we just switch from SSG to SSR with npm astro add node for example, immediately we will see that no validation is run when running npm run build. Let's force it by importing the validation file in astro.config.ts:

import "./src/env-validate";

// ... or if delaying the validation for later on in the file:
await import("./src/env-validate");

Now npm run build will tell us that our environment variables are nowhere to be found in import.meta.env. How comes?

With Astro SSR, you have 1 chance at running the validation at build time, which is the moment when astro.config.ts is evaluated at the beginning of the build. Unfortunately import.meta.env at this "config reading time" is not populated yet, so you have to fallback to process.env or reading the .env files manually with Vite's loadEnv function. Thus putting import.meta.env into runtimeEnv doesn't work anymore with Astro SSR.

When working with Astro SSR, because there are the build-time and the runtime, there exists different types of environment variables. Here is one way to do it in Astro SSR, Node.js runtime. Note that each runtime has different ways of configuring t3-env, I'm just providing one of the Node runtime solutions.

// astro.config.ts

// ...

// Formulate the BUILDVERSION env var if not available
if (!process.env.BUILDVERSION) {
  console.log("Generating build version...");
  const lastCommitHash = execSync("git rev-parse --short HEAD")
    .toString()
    .slice(0, -1);
  const lastCommitTime = Math.floor(
    Number(
      new Date(execSync("git log -1 --format=%cd ").toString()).getTime() /
        60000
    )
  );
  process.env.BUILDVERSION = `${lastCommitTime}/${lastCommitHash}`;
}

// Assign .env to the fileEnv object. See fileEnv.ts for more details.
Object.assign(fileEnv, loadEnv(import.meta.env.MODE, process.cwd(), ""));

// Validate the environment variables
await import("./src/t3-env");

export default defineConfig({
  // ...
  output: "server",
  adapter: node({
    // ...
  }),
  vite: {
    define: {
      "process.env.BUILDVERSION": JSON.stringify(process.env.BUILDVERSION),
    },
  },
});

// fileEnv.ts
// (Server-side only) This file contains an exported object that will be
// assigned the content of the .env files (if any) by the astro.config.mjs file.
//
// The purpose is to use variables in the .env file in the
// ./src/env-validate.ts file without that file having to use Vite's `loadEnv`
// function, causing the `fsevents` module to be bundled in the client-side
// bundle.
const fileEnv: Record<string, string> = {};

export default fileEnv;

// t3-env.ts
import { createEnv } from "@t3-oss/env-core";
import { z } from "zod";

import fileEnv from "./fileEnv";

const isServer = typeof window === "undefined";
const isBuilding = isServer && process.env.BUILDING === "true";

export const serverValidation = {
  build: {
    // Put in here any server-side variable that is hardcoded on build, i.e.
    // defined with Vite config's `define` option.
    BUILDVERSION: z.string().nonempty(),
  },
  runtime: {
    // Put in here any server-side variable that is read at runtime, i.e. being
    // read from `process.env`.
    NODE_ENV: z.enum(["development", "test", "production"]),
    AUTOCOMPLETE_HOST: z.string().nonempty(),
    API_HOST: z.string().nonempty(),
    APP_DOMAIN: z.string().nonempty(),
  },
};

export const env = createEnv({
  server: (isBuilding
    ? serverValidation.build
    : {
        ...serverValidation.build,
        ...serverValidation.runtime,
      }) as typeof serverValidation.build & typeof serverValidation.runtime,

  clientPrefix: "PUBLIC_",
  // The client variables are hardcoded at build time and can only be read from
  // `import\u002meta.env`. No `process.env` available on client - there is no
  // runtime client variable.
  client: {
    PUBLIC_EXAMPLE_HARDCODED_CLIENT_VAR: z.string().nonempty(),
  },

  runtimeEnv: {
    ...fileEnv, // For development, loading from .env and the environment

    // After build, import\u002meta.env contains:
    // - Client variables
    // - (Only on the server version of the bundle) Any env var that is present
    // in the code and is indeed present in the environment (at the build
    // time). If it is used in the code but doesn't exist in the environment,
    // it won't be included here. For example, if the environment has a
    // variable named `VARIABLE` and in the code there is an exact match of
    // `VARIABLE` (even `abcVARIABLEabc`), then this object will be like `{
    // VARIABLE: process.env.VARIABLE }`.
    //
    // Example after build:
    // ```
    // // Server version (by Astro's vite-plugin-env)
    // Object.assign({ PUBLIC_VAR: "abc"}, { BUILDING: process.env.BUILDING })`
    //
    // // Client version (by Vite)
    // { PUBLIC_VAR: "abc" }
    // ```
    //
    // See https://github.com/withastro/astro/blob/main/packages/astro/src/vite-plugin-env/index.ts
    //
    // The purpose is to continue fetching these vars from the environment at
    // runtime.
    //
    // Note that if the variables are:
    // - In the .env file (instead of in the environment), or
    // - Defined in the Vite's `define` option with `process.env.XXX`
    // then they will be included in the bundle at build time. The reason for
    // the former is... they are defined that way by Astro's vite-plugin-env.
    // The reason for the latter is, after it becomes `process.env.XXX`, Vite's
    // `define` will go and replace that with something else.
    //
    // Example: Server bundle after build (note the `BUILDVERSION`):
    // ```
    // Object.assign({ PUBLIC_VAR: "abc"}, { BUILDVERSION: "asdasd", BUILDING: process.env.BUILDING })`.
    // ```
    //
    // As for our use case, we use this object for client variables and
    // build-time server-side variables (by defining `process.env.XXX` via
    // Vite's `define`).
    ...import.meta.env,

    // The above object just provides client-side variables and build-time
    // server-side variables. The below is the runtime environment, providing
    // runtime server-side variables.
    //
    // We don't want the client to touch `process`, hence the `isServer`.
    ...(isServer ? process.env : {}),
  },
});

[alexanderniebuhr]

Thank you for the write-up. I had to read it 3 times, before I understood it, and most things work. However in Cloudflare Workers, I do have issues setting up server-side runtime vars. The process.env is available and provides the vars, however it seems that in the build-step process.env will be replaced by {}, in the createEnv(). Have you made it work with Cloudflare and Astro?

CF_PAGES_URL: isServer ? {}.CF_PAGES_URL : void 0,

[intagaming]

Thank you for the write-up. I had to read it 3 times, before I understood it, and most things work. However in Cloudflare Workers, I do have issues setting up server-side runtime vars. The process.env is available and provides the vars, however it seems that in the build-step process.env will be replaced by {}, in the createEnv(). Have you made it work with Cloudflare and Astro?

CF_PAGES_URL: isServer ? {}.CF_PAGES_URL : void 0,

Weird, since the Astro build step has nothing to do with Cloudflare Workers.

Can you run the Astro build script locally and confirm that the process.env is indeed being replaced by Astro? Also can you confirm that Cloudflare's build step doesn't do anything special other than running the build script?

[alexanderniebuhr]

My bad explaining. I'm running the local build step astro build, but with the @astrojs/cloudflare adapter. And the adapter does the replacement.
However on Cloudflare Pages the env variables are not available on start, only after the request is received. That means that validation will fail on start of the runtime.

So I made everything work, except server-side runtime vars. Not sure what it takes to make that work with Cloudflare Pages, the @astrojs/node is much easier.

Input

//t3-env.ts
const serverValidation = {
	build: {
		// Put in here any server-side variable that is hardcoded on build, i.e.
		// defined with Vite config's `define` option.
		TIME: z.string().nonempty(),
	},
	runtime: {
		// Put in here any server-side variable that is read at runtime, i.e. being
		// read from `process.env`.
		CF_PAGES_URL: z.string().nonempty(),
	},
}
export const env = createEnv({
	server: (isBuilding
		? serverValidation.build
		: {
				...serverValidation.build,
				...serverValidation.runtime,
		  }) as typeof serverValidation.build & typeof serverValidation.runtime,
	clientPrefix: "PUBLIC_",
	client: {
		PUBLIC_API_URL: z.string(),
	},
	// We can't use vite's import meta env here, because it's not loaded yet
	runtimeEnvStrict: {
		TIME: isServer ? process.env.TIME : undefined,
		CF_PAGES_URL: isServer ? process.env.CF_PAGES_URL ?? devEnv("CF_PAGES_URL") : undefined,
		PUBLIC_API_URL: isBuilding ? process.env.PUBLIC_API_URL : import.meta.env.PUBLIC_API_URL ?? fileEnv.PUBLIC_API_URL,
	},
	skipValidation:
		!!process.env.SKIP_ENV_VALIDATION &&
		process.env.SKIP_ENV_VALIDATION !== "false" &&
		process.env.SKIP_ENV_VALIDATION !== "0",
})

Output

//_workers.js
var env = g({
  server: isBuilding ? serverValidation.build : {
    ...serverValidation.build,
    ...serverValidation.runtime
  },
  clientPrefix: "PUBLIC_",
  client: {
    PUBLIC_API_URL: z.string()
  },
  // We can't use vite's import meta env here, because it's not loaded yet
  runtimeEnvStrict: {
    TIME: isServer ? "1234" : void 0,
    CF_PAGES_URL: isServer ? {}.CF_PAGES_URL ?? devEnv("CF_PAGES_URL") : void 0,
    PUBLIC_API_URL: isBuilding ? {}.PUBLIC_API_URL : "http://localhost:8788/trpc"
  },
  skipValidation: !!{}.SKIP_ENV_VALIDATION && {}.SKIP_ENV_VALIDATION !== "false" && {}.SKIP_ENV_VALIDATION !== "0"
});

[intagaming]

@alexanderniebuhr If I'm not mistaken, @astrojs/cloudflare is using import.meta.env as the place for runtime environment variable, yes? Then using process.env in runtimeEnvStrict here is actually incorrect. process.env should be import.meta.env (for server-side runtime env vars that is). Can you try that and report back to see if that's the culprit?

Found this in _worker.js that leads me to the conclusion: https://github.com/withastro/astro/blob/main/packages/integrations/cloudflare/src/util.ts

[alexanderniebuhr]

@astrojs/cloudflare is using import.meta.env as the place for runtime environment variable

Kinda you are right (and that might be the issue here), because the runtime variables are set in Cloudflare after the build is finished. So really astro has nothing to do with it, and can't know them at build-time.
And in the Cloudflare Pages runtime, both are available. The variables you set in the dashboard or wrangler cli, are in process.env, as far as I debugged it. So we don't know server-side runtime variables at build-time.

If replaced with import.meta.env, it still does not work. Even with custom loadEnv. It is also used in the old code already by devEnv.

If I replace the {}, with process.env in the build output it seems to work, I've asked in the Astro discord if, and update this accordingly.

[intagaming]

Actually I don't see any import.meta.env after build, because Vite is supposed to replace those statically at build time. @astrojs/cloudflare as far as I can see doesn't have anything to do with import.meta.env.

Cloudflare's runtime environment variables are not in process.env but in the Cloudflare's request context. See https://developers.cloudflare.com/pages/platform/functions/bindings/#environment-variables and https://docs.astro.build/en/guides/integrations-guide/cloudflare/#access-to-the-cloudflare-runtime

What I suspect is that you kinda have to pass getRuntime().env to t3-env's runtimeEnv somehow for each request. If I understand correctly, each request would have different set of environment variables, so you can't use the same env validation for every requests to a Worker. https://developers.cloudflare.com/workers/learning/how-workers-works/

[alexanderniebuhr]

pass getRuntime().env to t3-env's runtimeEnv somehow for each request

That is correct, however not sure if that is possible & if that would work on initial start before a request is send.

Cloudflare's runtime environment variables are not in process.env

Actually they are accessible by process.env, so either that is an alias or the documentation is for workers only and not pages.

See this example, sadly import.meta.env get's replaced in the src code.. (https://f048ceb6.test-ef9.pages.dev/)

[intagaming]

pass getRuntime().env to t3-env's runtimeEnv somehow for each request

That is correct, however not sure if that is possible & if that would work on initial start before a request is send.

Cloudflare's runtime environment variables are not in process.env

Actually they are accessible by process.env, so either that is an alias or the documentation is for workers only and not pages.

See this example, sadly import.meta.env get's replaced in the src code.. (https://f048ceb6.test-ef9.pages.dev/)

Yes, all of that is correct. Though it is certainly possible. You are given a little space on a Worker to perform tasks, which means each of your request has a different environment, which means you have to run the validation per request i.e. at the start of each individual request, not at initial start. There's no initial start in Cloudflare Workers, the workers are not your app's, it is shared between possibly thousands of apps.

Move createEnv into a function, say createT3Env. At the beginning of your Astro server code, instead of importing env, you call createT3Env(getRuntime(Astro.request).env); use that input as the replacement for the usual process.env.

[alexanderniebuhr]

So for me to make it work with Cloudflare Pages SSR it is, the changes here!

⚠️ This will leak variables inside client islands, if env is imported inside them.

I personally thing Astro support t3-env is far from ideal, IMO following is still missing (to be edited):

• Throw if non-client variables are leaked inside props. Obviously this allows server variables if code is run on server to generate HTML. However now I can just add a server variable as an prop for an UI Component and it will show up in plain text in the HTML. This is an leak, which should be prevented.
• If I import the validation function inside a UI Component and then use client:only island, I will get errors and no render if I try to access an server variable, which is expected. But the validation function is in the js island chunk. So some can open dev tools, look at the js chunk and find the variables.

UPDATE
It is possible to support Astro SSR with less work and also with one less leaking issue. You need, the following files. And if you want to use it you just call the function(const env = createRuntimeEnv(getRuntime(Astro.request)?.env);) with either import.meta.env / getRuntime(Astro.request).env:

// t3-env.ts (the file you use for validation)
import { createEnv } from "@t3-oss/env-core"
import { z } from "zod"

const isBuild = process?.env?.STATE === "building"

const serverVariables = {
	build: {
		// Put in here any server-side variable that is hardcoded on build
		VITE_TIME: z.string().nonempty(),
	},
	runtime: {
		// Put in here any server-side variable that is read at runtime
		CF_PAGES_URL: z.string().nonempty(),
	},
}

export const createRuntimeEnv = (prop?: unknown) => {
	const rEnv = prop as Record<string, string | number | boolean | undefined>
	return createEnv({
		skipValidation:
			!!process.env.SKIP_ENV_VALIDATION &&
			process.env.SKIP_ENV_VALIDATION !== "false" &&
			process.env.SKIP_ENV_VALIDATION !== "0",
		server: isBuild ? serverVariables.build : { ...serverVariables.runtime, ...serverVariables.build },
		clientPrefix: "PUBLIC_",
		client: {
			PUBLIC_API_URL: z.string(),
		},
		// check if rEnv is empty, if so use only import.meta.env, otherwise combine with env from runtime / build
		runtimeEnv: rEnv ? { ...rEnv, ...import.meta.env } : import.meta.env,
	})
}

// astro.config.ts
import { loadEnv } from "vite"

try {
	await import("./src/t3-env").then((m) => m.createRuntimeEnv(loadEnv(import.meta.env.MODE, process.cwd(), "")))
} catch (error) {
	console.error(error)
	process.exit(1)
}
// https://astro.build/config
export default defineConfig({
	vite: {
		define: {
			"process.env.VITE_TIME": JSON.stringify(process.env.VITE_TIME),
		},
	},
})

[intagaming]

I updated the original issue with my new setup. There were some misunderstandings of the Vite build step, so I documented the correct behavior in the code.

As for @alexanderniebuhr's solution:

• Beware that the solution is supposed to be a solution for Cloudflare Pages. Calling createRuntimeEnv for each file, or function, is wasteful if the argument doesn't change. process.env in the Node runtime doesn't change, but getRuntime(Astro.request)?.env in Cloudflare Pages does. Applying the solution of createRuntimeEnv for Node runtime is overkill and will lead to wasteful CPU cycles.

• In my solution, because I need to loadEnv before I import t3-env.ts, I have to loadEnv into an object somewhere so t3-env.ts can read from there. Definitely not astro.config.ts because loading astro.config.ts on the client-side doesn't make sense. Hence the fileEnv.ts instead of passing loadEnv into createRuntimeEnv.

• If you don't include as typeof serverValidation.build & typeof serverValidation.runtime, you won't have TypeScript completion for the .runtime object.

[alexanderniebuhr]

Agree! There are different solutions for different adapters, and your new one looks great for Node runtimes. We should maybe consider splitting this issue then :)