Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firebase functions emulator does not work with Yarn 2 #2198

Closed
cjmyles opened this issue May 3, 2020 · 16 comments
Closed

Firebase functions emulator does not work with Yarn 2 #2198

cjmyles opened this issue May 3, 2020 · 16 comments
Assignees

Comments

@cjmyles
Copy link

cjmyles commented May 3, 2020

[REQUIRED] Environment info

firebase-tools:
8.2.0

Platform:
macOS

Node:
10.16.0

Yarn:
2.0.0-rc.33

[REQUIRED] Test case

When generating, installing and running a fresh Typescript version of a functions project using the firebase-tools CLI tool and Yarn 2, 20 errors are produced when executing yarn serve - 3 unique:

Cannot find module 'express-serve-static-core'.
Cannot find type definition file for 'node'.
'functions' is declared but its value is never read.

I can resolve these in the following way, but the resolution causes other errors which are not curable:

  1. yarn add @types/node -D
  2. yarn add @types/express-serve-static-core -D
  3. Modify src/index.ts to export the helloWorld function present in the comments.

This causes the following CLI output:

i  Starting emulators: ["functions"]
✔  functions: Emulator started at http://localhost:5001
i  functions: Watching "/Users/craig/dev/sandbox/firebase-functions-yarn-2" for Cloud Functions...
⚠  Error: Cannot find module '/Users/craig/dev/sandbox/firebase-functions-yarn-2/.yarn/cache/firebase-admin-npm-8.11.0-40c4d739ef-3.zip/node_modules/firebase-admin/package.json'
Require stack:
- /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js
Require stack:
- /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:957:15)
    at Function.module_1.Module._resolveFilename (/Users/craig/dev/sandbox/firebase-functions-yarn-2/.pnp.js:12283:54)
    at Module._load (internal/modules/cjs/loader.js:840:27)
    at Function.module_1.Module._load (/Users/craig/dev/sandbox/firebase-functions-yarn-2/.pnp.js:12160:36)
    at Module.require (internal/modules/cjs/loader.js:1019:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:123:32
    at Generator.next (<anonymous>)
    at fulfilled (/usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:4:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
⚠  Your function was killed because it raised an unhandled error.

I can confirm that the zip file '/Users/craig/dev/sandbox/firebase-functions-yarn-2/.yarn/cache/firebase-admin-npm-8.11.0-40c4d739ef-3.zip' contains the path 'node_modules/firebase-admin/package.json'.

I tried downgrading to firebase-tools v8.0.3 as per this issue to no avail.

I tried upgrading Yarn to use the latest sources version to no avail.

I tried using a newer version of Node (v12.16.3) to no avail.

I tried using a newer version of tslint (v6.1.2) to no avail.

Could be an issue with the PnP resolver (https://classic.yarnpkg.com/en/docs/pnp/troubleshooting)? Although I tried running the emulator using the following command, and the error still prevailed.

NODE_OPTIONS="--require ./.pnp.js" firebase emulators:start --only functions

I've created a repository containing the generated project so it's as minimal as possible: https://bitbucket.org/cjmyles/firebase-functions-yarn-2/src. I've included firebase-debug.log as well as the Yarn 2 Zero Install files.

[REQUIRED] Steps to reproduce

  1. Install Yarn 2.
  2. Execute firebase init functions from the CLI - choose a Firebase project; Typescript (Y); TSLint (Y); install dependencies (N).
  3. Execute yarn to install dependencies.
  4. Execute yarn serve to see the first set of errors.

[REQUIRED] Expected behavior

Generated project should execute without issues, same as if the project's dependencies were installed and the project executed using Yarn 1.

[REQUIRED] Actual behavior

See test case above.

@cjmyles cjmyles changed the title CLI generated functions project does not work with Yarn 2 Firebase functions emulator does not work with Yarn 2 May 3, 2020
@samtstern
Copy link
Contributor

Thank you @cjmyles for the really detailed feedback. This is all way over my level of JS knowledge, I'm hoping @abeisgoat can chime in.

@cjmyles
Copy link
Author

cjmyles commented May 5, 2020

Thanks @samtstern. Interestingly, as soon as I add express to the mix (as per this basic example), I get another error:

../../.yarn/$$virtual/firebase-functions-virtual-bbad2b487f/0/cache/firebase-functions-npm-3.6.1-9aae15f59c-3.zip/node_modules/firebase-functions/lib/handler-builder.d.ts:21:123 - error TS2307: Cannot find module 'express-serve-static-core'.

Running yarn add @types/express @types/express-serve-static-core -D doesn't fix the issue.

So a few Yarn 2 related issues here. I'd like to use Yarn 2 as it resolves transitive dependencies correctly (I have a React/Functions monorepo for example using Yarn workspaces, which references "core" packages on my machine and Yarn 1 has link issues with libraries such as React which need to exist singularly but still be referenced in both), it's a lot quicker than Yarn 1 and offers some other nice features. I'd imagine a lot of people will be migrating over.

@cjmyles
Copy link
Author

cjmyles commented May 10, 2020

@samtstern @abeisgoat Any ideas?

@abeisgoat
Copy link
Contributor

Will take a look this week and report back. I'm not familiar with the differences between yarn and yarn@2 so it'll take a moment I'm sure

@cjmyles
Copy link
Author

cjmyles commented May 19, 2020

@abeisgoat did you get a chance to take a look? Yarn 2 is a life saver for me in terms of the transitive dependency resolution feature - I have a core React library that can be imported into multiple other React projects no problem, all running within Workspaces. The missing piece of the jigsaw is getting Firebase to run properly in the same setup, so any help would be greatly appreciated.

@cjmyles
Copy link
Author

cjmyles commented Jun 11, 2020

@samtstern @abeisgoat any update on this? Or is Yarn 2 something that shouldn't be used with Firebase?

@samtstern
Copy link
Contributor

@cjmyles I'll take a look at this today. We do a lot of manual module resolution in the emulator and I expect that PNP is tripping us up.

@samtstern
Copy link
Contributor

Yikes PNP is doing some really nasty stuff, like patching fs so it looks inside ZIPs:

https://yarnpkg.com/features/pnp#packages-are-stored-inside-zip-archives-how-can-i-access-their-files

@samtstern
Copy link
Contributor

Ok I got realllllly far with this but yarn got in my way at the very end:
yarnpkg/berry#1467

I have a few workarounds possible but none are great so I would like for the Yarn2 team to get back to me first.

@samtstern
Copy link
Contributor

@cjmyles I have a work in progress here:
#2356

Could you try this:

  1. Clone that branch
  2. Run npm install && npm build && npm link to install my branch of the Firebase CLI as the firebase command on your machine
  3. See if you're able to load your Yarn2 project in the emulator?

@samtstern
Copy link
Contributor

After a bunch of discussion in #2356 we realized that there's no safe way for us to support Yarn2 with PnP in the local emulators that won't break in production. But we're going to add a helpful warning to make this less confusing.

@cjmyles
Copy link
Author

cjmyles commented Feb 20, 2021

@samtstern Apologies for the radio silence. Work commitments got the better of me. Thanks for debugging the issue and the work on the PRs, especially adding the warning, it's greatly appreciated. I'm tempted to try a Lerna workspace with Yarn 1 or NPM to see if it fits the setup I need and avoids the issues with Yarn 2.

This was referenced Mar 12, 2021
@ericvera
Copy link
Contributor

FYI, In case someone else ends up here to make the emulator run, instead of running firebase directly from the command line run it as a yarn command. After some reading, it seems yarn takes care of virtualizing node_modules if necessary.

Two things are needed:

  1. Latest yarn yarn set version from sources

  2. In package.json

{
  ...
  "scripts": {
    ...
    "start": "firebase emulators:start --project project-name"
    ...
  },
  "dependencies": {
    ...
    "exegesis": "^2.5.6",
    "firebase-tools": "^9.6.1",
    "typescript": "^4.2.3",
    "winston-transport": "^4.4.0"
  },
}

@aaronncfca
Copy link

@ericvera I did end up here trying to make the emulator run, but I was already using the yarn serve command, which per the initial package.json resolves to firebase emulators:start --only functions. My yarn version is 3.2.1. Still, yarn serve produces the error:

Error: Error parsing triggers: Cannot find module 'firebase-functions'

Is it possible that new versions of yarn/Firebase CLI no longer work with this? Or perhaps it's just because my code is set up in a monorepo? (I have tried with and without setting installConfig.hoistingLimits as shown below, to keep yarn from hoisting packages out of the functions directory up to the root).

In any case, for me (specifically a yarn monorepo with yarn 3.2.1) I was only able to proceed by disabling Yarn PnP for the functions workspace (setting nodeLinker: node-modules in the functions/.yarnrc.yml file), and I set the following in functions/package.json to keep it from hoisting anything to the root directory:

 "installConfig": {
    "hoistingLimits": "workspaces"
  }

@yuchenshi
Copy link
Member

@aaronncfca Would you mind opening a new issue with more details?

@ericvera
Copy link
Contributor

@aaronncfca

If you are using yarn PnP checkout #4275. I did not have the time to create a smaller repro so that issue it was closed. It seems to be specific to the functions emulator (auth and Firestore emulators run fine using yarn commands). I was able to get around the issue by installing firebase-tools globally using npm and running outside of yarn. Not ideal, but works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants