Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add firebase authentication emulator to emulator suite #1677

Open
vdjurdjevic opened this issue Sep 26, 2019 · 24 comments
Open

Add firebase authentication emulator to emulator suite #1677

vdjurdjevic opened this issue Sep 26, 2019 · 24 comments

Comments

@vdjurdjevic
Copy link

@vdjurdjevic vdjurdjevic commented Sep 26, 2019

Is it possible to emulate the authentication API for faster local development and end to end testing? With the Firestore emulator, we don't have to create different projects for different environments (dev, testing, staging, prod), we can just use emulator, seed data to it from JSON files in watch mode for development, or seeding for every test case, etc. To be able to write end to end test that has a user log in, navigation based on role, and then some action, we have to create a separate project to isolate test users, and also seed and clear users there for every test case.

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Sep 30, 2019

@vladimirdjurdjevic thanks for raising this! This is on our list of things to do, but we have not yet decided on how much of the auth service to emulate and the best strategy to do it. Certain things (like anonymous auth) are very easy to emulate while others (like SMS authentication) are very hard.

But we definitely want to enable the end-to-end use case you mentioned!

Question: would you find it useful if there was a library that allowed you to locally mock a Firebase user for use with the emulators? Something like:

firebase.auth().setEmulatedUser({
   uid: 'abc123',
   // ...
});
@noelmansour

This comment has been minimized.

Copy link

@noelmansour noelmansour commented Oct 2, 2019

@samtstern Would the call to setEmulatedUser() also trigger any functions.auth.user().onCreate() emulated cloud functions? If so, that would be very useful.

@vdjurdjevic

This comment has been minimized.

Copy link
Author

@vdjurdjevic vdjurdjevic commented Oct 2, 2019

@samtstern That would help some testing scenarios for sure, but still require real instance for development. My idea is to avoid creating multiple firebase projects for different environments, and to be able to develop locally (offline perhaps). Another approach would be to support different environments for services. If we could create different environments for let's say firestore and auth under the same project, it would solve many issues. I know I can create project per environment, but that's a real pain in the ass. Configuring every environment, replicating data, etc. Ideally, I would like to be able to create one firebase project, and if I need dummy data for manual testing, I could just create a staging environment for firestore, and upload data there.

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Oct 3, 2019

@noelmansour good question! That wouldn't be too hard to do, we'd probably want two different calls like signInAsEmulatedUser and signUpAsEmulatedUser where only the second triggers the function.

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Oct 3, 2019

@vladimirdjurdjevic totally agree that a full-featured emulator is best. Could you explain what things you would need from a "real instance for development" that are not solved by being able to set the user locally?

@IlCallo

This comment has been minimized.

Copy link

@IlCallo IlCallo commented Oct 9, 2019

Question: would you find it useful if there was a library that allowed you to locally mock a Firebase user for use with the emulators?

Really useful, it would help a lot.
Right now I'm just setting to true the rule function validating logged user every time I have to test something locally, it's pretty unsafe yet probably the simpler thing I can do without emulated current user.

@DGmip

This comment has been minimized.

Copy link

@DGmip DGmip commented Oct 10, 2019

Yes this would be incredibly useful

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 13, 2019

I also would like to unit test functions.auth.user().onCreate(). I suppose right now the best workaround is essentially export callback function passed to onCreate and supply fake user object to it.

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Oct 13, 2019

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 13, 2019

@samtstern That actually works along with Firestore emulator? Base on this I got the impression it's for a different purposes.

image
https://firebase.google.com/docs/functions/unit-testing#initializing

I certainly don't want online mode for unit testing, that would be a slowpoke. And I am not sure if I can access the Firestore emulator by "stubbing".

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Oct 14, 2019

@FredyC thanks for pointing out those docs. The words used are confusing because the "modes" are not a switch you can enable, they're just describing strategies you can take.

If you were to start up the Firestore emulator alongside your unit tests, your code could definitely connect to it. If you set the FIRESTORE_EMULATOR_HOST environment variable the Firebase Admin SDK will automatically connect to the Firestore emulator (firebase emulators:exec does this for you).

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 16, 2019

@samtstern I still have trouble connecting dots and how is all that going to help me to test functions.auth.user().onCreate()? I mean it's great that functions will connect to emulator instead of a production version, but that's only Firestore, right? How do I invoke user creation from tests to actually have the function code to kick in?

There seems to be some cryptic method makeUserRecord in the mentioned firebase-functions-test, but it doesn't make much sense how would that work or how to actually use it.

I tried calling auth.createUserWithEmailAndPassword() from the @firebase/testing package, but that's complaining about invalid API key, so I assume it would work with online version only.

When I search through org for that env variable you have mentioned, it's only in three places and none seems to be relevant to auth really. Unless it's hidden by some string concatenation.

I've been also browsing through the https://github.com/firebase/functions-samples but I found no examples of unit testing there.

Can you please make some sense of this? :)

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 16, 2019

I also have another case, somewhat opposite, where the cloud function code is using admin.auth().getUserByEmail() call. Surprisingly it doesn't end with the error, but I have no idea how I would create that user so it can be returned. Except of course mocking out the whole admin module, but that's crazy :)

@vdjurdjevic

This comment has been minimized.

Copy link
Author

@vdjurdjevic vdjurdjevic commented Oct 16, 2019

@samtstern Sorry for the delay. When I say real instance, I mean real instance :D Ideally, I just want to swap config in my angular environment file for development, and to keep implementing auth features like my app is talking to real API, but it actually emulator running on my machine, and I can do it offline. Now, I understand challenges with sending SMS for example. It would be silly to expect the emulator to send real SMS to my phone, but you could maybe just print it to console (the content of SMS that would be sent). It's probably more hassle than the value with this. That's why I think that just supporting multiple environments per project could make stuff much better. It takes to much time to replicate configs between multiple projects for different environments. And also managing multiple service accounts for scripts to be able to seed data to different Firestore projects is a pain. That's why I taught that if we had a whole firebase platform emulated, we could use that for every non-production environment, and manage just one real firebase project. But maybe just supporting multiple environments per project is a better solution with an acceptable outcome.

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Oct 16, 2019

@FredyC thanks for all your feedback! It's really helpful to us to see how confusing this can be. There are two main things people want to do in their tests related to auth:

  1. Sign in to use services like Firestore or Realtime Database without actually creating real users. Right now that's what something like setEmulatedUser would solve. It would just allow you to have a bogus auth token locally that the emulators would accept but it would be rejected by prod. More safety, more isolation, etc.
  2. Actually test authentication directly. This would have a few pieces:
    a. An auth emulator that responds to all the necessary API calls so that you can point the Auth SDKs at it.
    b. Integration between that emulator and the functions emulator so that .onCreate functions are triggered correctly.
    c. Auto-mocking inside the functions emulator so that admin.auth() points to the Auth emulator, just like we do for Firestore and RTDB today.

I think clearly (2) is the right story but I was trying to get a sense of how many people would be happy with (1) since it's substantially simpler to implement and maintain.

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 16, 2019

@samtstern I see. Correct me if I am wrong, but isn't the (1) already solved? I mean in tests I can just call the following and Firestore emulator will recognize me as that user so I can test against security rules. I haven't actually tried that yet, but looks promising so far :)

  const app = firebase.initializeTestApp({
    projectId,
    auth: {
      uid: 'owner'
    }
  })

The (2) definitely sounds very useful, but a lot more complex to tackle for sure. Sadly, my insight into the full product is so limited, I cannot really offer any useful ideas here.

I think it should be built incrementally. Instead of trying to cover all scenarios at once, build it based on known use cases and adding on the go. In my limited opinion emulating "user database" along with function triggers don't seem that hard to do.

@samtstern

This comment has been minimized.

Copy link
Member

@samtstern samtstern commented Oct 16, 2019

@FredyC you're right that (1) is solved for use inside test suites. But the other use case is actually connecting your Android/iOS/Web app directly to the emulator for local development. In which case you can't use @firebase/testing.

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 17, 2019

I see. Honestly, it would be kinda superb if @firebase/testing could be used cross-platform instead of having separate solutions. I mean how hard it can be to redirect communication to the emulator? Isn't the FIRESTORE_EMULATOR_HOST for exactly that? Although I think something like FIREBASE_EMULATOR_HOST would be more appropriate if the emulator is going to have other services as well.

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 17, 2019

@vladimirdjurdjevic I am thinking, wouldn't actually work to basically mock out the signInWithPhone method so you can control its behavior? Then you don't need to worry about an emulator and getting simulated SMS code in the console :)

Of course, then you need some way to authenticate toward the emulator for Firestore (and connect to it). Something like I outlined in previous #1677 (comment). There is an underlying method for generating unsecured tokens: https://github.com/firebase/firebase-js-sdk/blob/master/packages/testing/src/api/index.ts#L64. Not sure if that would work.

Of course, mocking 3rd party libraries is not always that easy, but once figured out, it can bring great results. Eventually, it could be extracted to some library to make it generally easier.

I am also thinking these signIn methods can throw quite a plethora of error codes which is something proper tests should take care of as well. That's easy to do with mocking as well.

@toddpi314

This comment has been minimized.

Copy link

@toddpi314 toddpi314 commented Oct 21, 2019

@samtstern Extending the firebase.auth() namespace context with something topical like setEmulatedUser seems like an anti-pattern with the existing emulation strategies. Is that recommendation perhaps influenced by ease-of-extensibility on the package side?

Inline with the other emulators, I would expect a separate authentication service tier to be launched over HTTP and a client-side config can redirect the existing API surface to utilize.

I would much rather have eccentric AuthN cases return errors with the Admin and Client API minimally supporting CRUD for basic users over username/password. Heck, i think even starting with Custom Token support in the Admin and signInWithCustomToken would go really far. Maybe implement low hanging fruit with an API support matrix published in the docs.

To @FredyC point, the current strategy for integration Auth testing is to conditionally import the @firebase/testing in application code to route to the custom initializeTestApp call. This action both stresses package exclusion at build-time for project teams, and also spreads the emulator redirect configs across two package APIs (initializeTestApp and firestore.settings/functions.useFunctionsEmulator)

Hack the planet!

@FredyC

This comment has been minimized.

Copy link

@FredyC FredyC commented Oct 21, 2019

the current strategy for integration Auth testing is to conditionally import the @firebase/testing in application code to route to the custom initializeTestApp call.

Um, I am calling that method inside tests. The trick is that regular initializeApp resides in index.ts which imports functions. It's being called when emulator starts, but when tests are executing, it's a different process and it doesn't conflict with each other. So there is really no burden of conditional import.

Of course, this might be different for testing auth in eg. web app where test runner would share the process with app code. However, with unit testing you don't usually import a whole app into a test. The initializeApp is probably done in some code that's not relevant to tests and is not imported at all.

@toddpi314

This comment has been minimized.

Copy link

@toddpi314 toddpi314 commented Oct 21, 2019

@FredyC Agreed on the documented usages for unit tests. Was really speaking to full-app scenarios where the apis diverge and dynamic imports go off the documented map.

I just wanted to give you attribution for Honestly, it would be kinda superb if @firebase/testing could be used cross-platform instead of having separate solutions. digital high five

@diginikkari

This comment has been minimized.

Copy link

@diginikkari diginikkari commented Oct 25, 2019

@FredyC thanks for all your feedback! It's really helpful to us to see how confusing this can be. There are two main things people want to do in their tests related to auth:

  1. Sign in to use services like Firestore or Realtime Database without actually creating real users. Right now that's what something like setEmulatedUser would solve. It would just allow you to have a bogus auth token locally that the emulators would accept but it would be rejected by prod. More safety, more isolation, etc.
  2. Actually test authentication directly. This would have a few pieces:
    a. An auth emulator that responds to all the necessary API calls so that you can point the Auth SDKs at it.
    b. Integration between that emulator and the functions emulator so that .onCreate functions are triggered correctly.
    c. Auto-mocking inside the functions emulator so that admin.auth() points to the Auth emulator, just like we do for Firestore and RTDB today.

I think clearly (2) is the right story but I was trying to get a sense of how many people would be happy with (1) since it's substantially simpler to implement and maintain.

@samtstern first of all I would ❤️to have this kind of emulation.

I would see no. 1 very useful for writing e2e tests. Currently I have to use real instance for authentication while I can use emulator for hosting, rules, firestore and functions.

I think it should be possible to use setEmulatedUser to mock user in same way it's done in firebase.initializeTestApp. It's should be possible to submit e.g. custom tokens and other user related data.

It should also be possible to get sample credentials which could be used in client app with firebase.auth().signInWithCredential(credential)

@dominikfoldi

This comment has been minimized.

Copy link

@dominikfoldi dominikfoldi commented Nov 13, 2019

Thank you @vladimirdjurdjevic for bringing up this issue! We were looking for a solution for almost a year already.

We would like to see a real emulator for three things:

  1. e2e tests for the whole app, so we do not have to create different environments as @vladimirdjurdjevic said.
  2. Integration tests for the backend, where the APIs are called and the user should be validated at Firebase.
  3. All of our developers are using a central Firebase environment during development, which causes lots of collisions. Of course, every developer could create their own Firebase project but they still need to manage their test users in the Firebase dashboard which is not ideal. Also, we would like to develop our app offline that is not possible today because of the lack of a real emulator.

I hope that you will release something for us soon, it would make your product more valuable! Developer productivity is very important in such services!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.