Use emulators only

README.md

@@ -2,16 +2,16 @@
 
 This is a quest for the ideal Typescript monorepo setup.
 
-My current projects are based on Node, Next.js, and Firebase, so that is what I
-am focussing on.
+My current projects are based on Node.js, Next.js, and Firebase, so that is what
+I am focussing on.
 
 If you use different platforms, this can still be a great starting point, as it
 should be easy to discard any packages that you have no use for. The monorepo
 approach by itself is largely independent of the chosen technology stack.
 
 This is meant as a best-effort approach given the tooling that is available, so
-I expect this code to change as the ecosystem around Typescript and Javascript
-continue to evolve.
+expect this repository to change as the ecosystem around Typescript and
+Javascript continue to evolve.
 
 Contributions are welcome within the scope of this example, but I doubt there
 will ever be a one-size-fits-all solution, so this code should be viewed as
@@ -27,22 +27,21 @@ opinionated.
 - [Features](#features)
 - [Install](#install)
 - [Usage](#usage)
-- [Workspace packages](#workspace-packages)
+- [Workspace](#workspace)
   - [Packages](#packages)
   - [Apps](#apps)
   - [Services](#services)
 - [Deployment](#deployment)
+- [Running Firebase using Emulators](#running-firebase-using-emulators)
 - [Using NPM instead of PNPM](#using-npm-instead-of-pnpm)
 - [Using Yarn instead of PNPM](#using-yarn-instead-of-pnpm)
 - [The "built packages" strategy](#the-built-packages-strategy)
   - [Convert path aliases](#convert-path-aliases)
-  - [Write ESM without import extensions](#write-esm-without-import-extensions)
+  - [Write ESM without import file extensions](#write-esm-without-import-file-extensions)
   - [Tree shaking](#tree-shaking)
-  - [](#)
 - [The "internal packages" strategy](#the-internal-packages-strategy)
 - [Live code changes from internal packages](#live-code-changes-from-internal-packages)
 - [Deploying to Firebase](#deploying-to-firebase)
-- [VSCode settings](#vscode-settings)
 
 <!-- /TOC -->
 
@@ -80,16 +79,28 @@ If you prefer to use a different package manager, that should not be a problem.
 See [using NPM](#using-npm-instead-of-pnpm) or
 [using Yarn](#using-yarn-instead-of-pnpm) for more info.
 
+> NOTE that at the moment, PNPM is the only package manager for which
+> [isolate-package](https://github.com/0x80/isolate-package/) will generate a
+> compatible lockfile for deployment. For other package managers the lockfile is
+> omitted and therefor deployments are not deterministic.
+
 ## Usage
 
-Run `pnpm dev`. This will:
+Run `npx turbo dev`
+
+This will:
+
+- Build the dependencies of the `web` app and start its dev server
+- Build the `api` and `fns` backend services and their dependencies
+- Start the Firebase emulators. See
+  [running Firebase emulators](#running-firebase-emulators) for more info
 
-- Build the dependencies of the web app and start its dev server
-- Build the backend services and their dependencies, and
-  [isolate](#deploying-to-firebase) the output
-- Start the backend emulators
+The web app should become available on http://localhost:3000 and the emulators
+UI on http://localhost:4000.
 
-## Workspace packages
+More info can be found in the README files of the various packages.
+
+## Workspace
 
 ### Packages
 
@@ -100,30 +111,47 @@ Run `pnpm dev`. This will:
 
 ### Apps
 
-- [web](./apps/web) A Next.js based web application.
+- [web](./apps/web) A Next.js based web application configured to use Tailwind
+  CSS and ShadCN components.
 
 ### Services
 
-- [fns](./services/fns) Cloud functions that execute on document writes, pubsub
-  events etc. This package shows how to set up the firebase.json config if you
-  would like to use isolate-package as part of your predeploy script. The
-  downside of this is that a watch task on your build does not result in live
-  code updates when running the emulator, because the results would then still
-  need to be isolated.
-- [api](./services/api) A 2nd gen (Cloud Run based) API endpoint, using Express.
-  This package shows how to use
+- [fns](./services/fns) Various Firebase functions that execute on document
+  writes, pubsub events etc. This package shows how to use [isolate-package]
+  explicitly as part of the predeploy phase.
+- [api](./services/api) A 2nd gen Firebase function (based on Cloud Run) serving
+  as an API endpoint, using Express. This package shows how to use
   [firebase-tools-with-isolate](https://github.com/0x80/firebase-tools-with-isolate)
-  in order to have the isolation process integrated as part of the
-  `firebase deploy` command. This does not interfere in any way and allows you
-  to use a build watch task to keep live code updates flowing to the emulator.
+  to have the isolation integrated as part of the `firebase deploy` command. In
+  addition it illustrates how to use secrets.
 
 ## Deployment
 
-Deployment instructions can be found in the individual packages:
+I consider deployment a bit out-of-scope for this demo.
+
+For deployment to Firebase you will have to set up and configure an actual
+project, but it is not required to run this demo since it can use the emulators.
+Additional info about the use of
+[isolate-package](https://github.com/0x80/isolate-package) (used by fns) and
+[firestore-tools-with-isolate](https://github.com/0x80/firebase-tools-with-isolate)
+(used by api) can be found in the instructions of each package.
+
+## Running Firebase using Emulators
+
+Throughout this repository we use the a Firebase demo project called
+`demo-mono-ts` which allows us to run emulators for the different components
+like database without actually creating any Firebase projects or resources.
+
+To make this work we pass the `--project` flag when starting the emulator. You
+can use any name that starts with `demo-`.
+
+When passing configuration to initializeApp you can use any non-empty string for
+the API keys as you can see in
+[apps/web/.env.development](apps/web/.env.development)
 
-- [web](./apps/web/README.md)
-- [fns](./services/fns/README.md#deployment)
-- [api](./services/api/README.md#deployment)
+Currently, if you want to make use of Firebase secrets,
+[you need to make sure they are also available in .env or .env.local](https://github.com/firebase/firebase-tools/issues/5520)
+for the emulators to work.
 
 ## Using NPM instead of PNPM
 
@@ -278,7 +306,33 @@ upload a self-contained package that can be treated similarly to an NPM package,
 by installing its dependencies and executing the main entry.
 
 This repo includes a solution based on
-[isolate-package](https://github.com/0x80/isolate-package/) and I encourage you
-to look at that and maybe read the
-[accompanying article](https://thijs-koerselman.medium.com/deploy-to-firebase-without-the-hacks-e685de39025e)
-to understand what it does and why it is needed.
+[isolate-package](https://github.com/0x80/isolate-package/). I wrote this
+[article](https://thijs-koerselman.medium.com/deploy-to-firebase-without-the-hacks-e685de39025e)
+explaining what it does and why it is needed.
+
+You might notice `@google-cloud/functions-framework` as a dependency in the
+service package even though it is not being used in code imports. It is
+currently required for Firebase to be able to deploy a PNPM workspace. Without
+it you will get an error asking you to install the dependency. I don't quite
+understand how the two are related, but it works.
+
+## Running Firebase Emulators
+
+For Firebase Functions each service (api and fns) start separate emulators on
+port 5001 and 5002. The backend serviced (using the firebase-admin api) connect
+to emulators by setting various environment variables.
+
+I have stored these in `.env` files in the respective service packages. Normally
+you would want to store them in a file that is not part of the repository like
+`.env.local` but by placing them in `.env` I prevent having to give instructions
+for setting them up just for running the demo.
+
+### Secrets
+
+The api services uses a secret for DEMO_API_KEY. In order to make secrets work
+with the emulator you currently have to add the secret to `.secret.local` and
+also an `.env` or `.env.local` file. See
+[this issue](https://github.com/firebase/firebase-tools/issues/5520) for more
+info. I have place it in `.env` which is part of the repo, so you don't have to
+set anything up, but .env.local is where you would put it normally, because that
+file is not checked into git.

apps/web/.env.development

@@ -0,0 +1,9 @@
+NEXT_PUBLIC_FIREBASE_API_KEY=any
+NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN=demo-mono-ts.firebaseapp.com
+NEXT_PUBLIC_FIREBASE_PROJECT_ID=demo-mono-ts
+NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET=demo-mono-ts.appspot.com
+NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID=12344567890
+NEXT_PUBLIC_FIREBASE_APP_ID=any
+NEXT_PUBLIC_DEMO_API_KEY=any
+NEXT_PUBLIC_DEMO_API_ENDPOINT="http://localhost:5002/demo-mono-ts/europe-west3/api/v1"
+NEXT_PUBLIC_USE_EMULATORS=true

apps/web/README.md

@@ -1,19 +1,14 @@
-This is a [Next.js](https://nextjs.org/) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app).
-
-## Getting Started
-
-First, run the development server:
-
-```bash
-pnpm dev
-```
+This is a [Next.js](https://nextjs.org/) project bootstrapped with
+[`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app).
 
 ## Deploy on Vercel
 
-Because this app uses Firebase, you'll need to create a project there first.
-Then go to project overview => Settings => General => Your Apps => Register web
-app => Add firebase SDK => Use Npm => ... and copy the different values to
-env variables in a `.env.local` file like so:
+Because this app uses Firebase, and we do not want to have to create actual
+Firebase resources just to run this code we use a so-called "demo" project. Any
+project name starting with `demo-` passed to the emulators will make them run
+without ever talking to real cloud resources.
+
+variables in a `.env.development` file like so:
 
 ```bash
 NEXT_PUBLIC_FIREBASE_API_KEY=
@@ -32,9 +27,19 @@ first deploy that.
 
 The value for `NEXT_PUBLIC_DEMO_API_ENDPOINT` should be
 `"http://localhost:5002/mono-ts/europe-west3/api/v1"` if you use the local
-firebase emulator. Otherwise it should point to your firebase function instance in the
-correct region.
+firebase emulator. Otherwise it should point to your firebase function instance
+in the correct region.
+
+The easiest way to deploy your Next.js app is to use the
+[Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme)
+from the creators of Next.js.
+
+Check out our
+[Next.js deployment documentation](https://nextjs.org/docs/deployment) for more
+details.
 
-The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
+## Run dev server
 
-Check out our [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details.
+```bash
+pnpm dev
+```

apps/web/app/components/key-value-list.tsx

@@ -14,7 +14,7 @@ export default function KeyValueList(props: {
   labels: Array<[string, string]>;
 }) {
   const rows = props.labels.map(([key, label]) => (
-    <TableRow className="bg-white">
+    <TableRow className="bg-white" key={key}>
       <TableCell className="px-4 py-3 text-sm font-medium text-gray-900">
         {label}
       </TableCell>

apps/web/app/page.tsx

@@ -1,23 +1,31 @@
 "use client";
 
 import { areWeThereYet } from "@mono/common";
+import { CardDescription } from "~/components/ui/card.jsx";
 import { add, multiply, reset } from "~/lib/api.js";
 import CardWithAction from "./components/card-with-action.jsx";
 import { CounterView } from "./components/counter-view.jsx";
 
 export default function Home() {
   return (
     <main className={"flex flex-col justify-between gap-10 p-10"}>
-      <h1 className={"text-xl"}>A quest for the ideal TS monorepo setup</h1>
+      <h1 className={"text-xl"}>A quest for the ideal TS monorepo</h1>
 
       <div className="grid grid-cols-2 gap-4">
         <CardWithAction
           title="Are we there yet? "
           description={areWeThereYet()}
-        />
+        >
+          <CardDescription>
+            To test live updates to the backend services while the emulator is
+            running, you can update code in `services/api/src/v1/handlers.ts`.
+            For example change the multiplication `counter.data.value * n` to
+            `counter.data.value * n * 10`
+          </CardDescription>
+        </CardWithAction>
         <CardWithAction
-          title="Remote Counter"
-          description="Implemented by mutating a Firestore document. Mutations will be slow as they flow through an API call and logic triggered by document onWrite events. Also, Firestore documents have a 1 write/sec limitation and so triggering mutations quickly could cause lagging or errors. This setup is aimed at integrating two separate Firebase services, and is not an example of how you would implement a reliable mechanism."
+          title="Dodgy Counter"
+          description="Implemented using a Firestore document. Mutations flow through an API endpoint and logic triggered by document onWrite events. In the emulator it works without noticeable lag but in real life it does not. This is a contrived example, aimed at demonstrating two separate Firebase services. It is not an illustration of how to implement an efficient and reliable mechanism."
           action={{ label: "Reset", handler: () => reset() }}
         >
           <CounterView counterId="my_counter" />

apps/web/lib/firebase.ts

@@ -1,5 +1,6 @@
 import { initializeApp } from "firebase/app";
-import { getFirestore } from "firebase/firestore";
+import { connectAuthEmulator, getAuth } from "firebase/auth";
+import { connectFirestoreEmulator, getFirestore } from "firebase/firestore";
 
 export const firebaseConfig = {
   apiKey: process.env.NEXT_PUBLIC_FIREBASE_API_KEY,
@@ -12,3 +13,9 @@ export const firebaseConfig = {
 
 export const app = initializeApp(firebaseConfig);
 export const db = getFirestore(app);
+export const auth = getAuth(app);
+
+if (process.env.NEXT_PUBLIC_USE_EMULATORS) {
+  connectFirestoreEmulator(db, "127.0.0.1", 8080);
+  connectAuthEmulator(auth, "http://127.0.0.1:9099");
+}

package.json

@@ -30,6 +30,7 @@
     "format": "prettier --write ."
   },
   "devDependencies": {
+    "concurrently": "^8.2.2",
     "prettier": "^3.1.0",
     "prettier-plugin-jsdoc": "^1.1.1",
     "turbo": "^1.10.16",