Storage emulator feature request

[nicobrinkkemper]

It's cool and all when github isn't the place for feature requests. But what if the feature request page isn't working?

We need a storage emulator so we can start testing our AutoML features locally. Are there any workarounds? Hacks? Thanks in advance,

Regards,

Jim

[bkendall]

PIng @samtstern as he has done a lot with the emulators.

Sorry to hear about the request page not working (and sorry for the super late response). It probably was a transient issue if you'd like to try again...

[samtstern]

Thanks for this! We are also interested in a Storage emulator, but no timeline to share right now.

[otri]

Hoping for some storage emulator love. ❤️ 🤖

[davigp]

I'd like to debug and test a storage function onFinalize, but seems it still not possible:

functions[generateThumbnail]: function ignored because the storage emulator does not exist or is not running.

How can I debug a onFinalize function locally without the storage emulator?
Is there any way to create unit tests for storage functions?

[samtstern]

@davigp you can use functions:shell to test those functions interactively or you can use the firebase-functions-test library to write unit tests.

[sebolio]

Any news on this? Sounds really cool

[sam-gc]

Hi folks, thanks for the interest! We're considering this, internal tracking bug b/159057998

[vanities]

@samtstern

functions-testing

can you link the repo? Do you mean the firebase-functions-test library?

[samtstern]

@vanities yes sorry that's what I meant, there are some examples here:
https://github.com/firebase/quickstart-testing

[vanities]

@samtstern
No worries, thanks again! Hoping we can get emulator testing for storage and functions, the emulator is so useful!

[IronDigitalMedia]

We are also interested in a storage emulator! Why wasn't this thought of? If so, if they couldn't do it why not just get ahead of it and post a blog post explaining the difficulties?

[tzoratto]

The Firestore emulator lets us test our security rules in a automated way on every build (https://firebase.google.com/docs/firestore/security/test-rules-emulator?hl=en) and on developer computers in one click, without any credentials or anything

Having the same workflow/feature for storage security rules would be awesome.

Automated and easy security testing is a game changer for me, I hope it will expand to storage as well 👍

[rdinkel]

We are also in need of the same awesome emulator testing feature for storage rules. Please push this request in its priority 💯 ;)

[warting]

+1 this would be awesome!

[HerrNiklasRaab]

@samtstern After authentication emulator, this one will be the next? :)

[valentinoavon]

I'd love a storage emulator! :D

[mhswork]

Firebase Storage is like the heart of firebase when we have projects dealing with storing images, videos etc. Once we get this in an Emulator then that would be a huge relief. Waiting for its release.

[ribomation]

It's awkward to handle two different credentials, one for the emulator suite and another one when accessing firebase storage.
What is the time frame for making the storage emulator available? A month from now, before end of year or first next year?

[samtstern]

@ribomation sorry we can't offer any timelines but I can say definitely longer than a month!

[chrisjacobs91]

Hi all - I would also love this emulator. In the meantime, is there a way I can avoid deleting live storage files and folders when running hosting / functions in the emulator?

For example I use the following code to delete all the files in a folder. Last night someone accidentally deleted all the documents in our emulator as part of a test and it deleted all the LIVE storage folders as we use an import of real documents into our emulator 🤦

async function deleteStorageFolder(path:string) {
  const bucket = admin.storage().bucket();
  return bucket.deleteFiles({
    prefix: path
  })

Is there any way I can tell firebase to avoid using the production storage APIs when emulators are running?

[itsravenous]

Is there any way I can tell firebase to avoid using the production storage APIs when emulators are running?

@chrisjacobs91 I use separate Firebase projects for dev/staging/ci. You don't have to get that granular, but having a non-prod project would help you avoid that.

But of course, really, I just want a storage emulator just like everyone else.

[Develrockment]

The storage emulator would be very helpful indeed

[premsuman8]

We need the storage emulator!
Give the people what they want!

[maranmaran]

+1

[jsduffy]

+1

[isamyh]

@samtstern

Any news? <3

[eladdad]

Is there a way to use Firebase Auth emulator with storage until there is a storage emulator? I need to emulate users uploading directly to storage with security rules applied

[bofeiw]

Storage emulator would be very useful to our automated testing. Hope it'll be available soon.

[bramosrys]

+1 Cannot test with security rules enabled because file access is denied due the Firebase Auth emulator running :(

[rip3rs]

Hi all, I'm here for the storage emulator :)

btw:

Is there any way I can tell firebase to avoid using the production storage APIs when emulators are running?

@chrisjacobs91 I use separate Firebase projects for dev/staging/ci. You don't have to get that granular, but having a non-prod project would help you avoid that.

But of course, really, I just want a storage emulator just like everyone else.

@itsravenous that can get expensive. Isn't that a bit dangerous? having multiple envs on firebase?

[itsravenous]

@itsravenous that can get expensive. Isn't that a bit dangerous? having multiple envs on firebase?

That's of course your call. I haven't scratched the surface of my free limit via automated tests run against my non-prod firebase env, but obviously that entirely depends on how many tests you have, how often you run them, and what they are doing 😄

I would certainly advise you set up billing alerts to ensure you don't spend more than you want to.

[bofeiw]

@itsravenous that can get expensive. Isn't that a bit dangerous? having multiple envs on firebase?

That's of course your call. I haven't scratched the surface of my free limit via automated tests run against my non-prod firebase env, but obviously that entirely depends on how many tests you have, how often you run them, and what they are doing 😄

I would certainly advise you set up billing alerts to ensure you don't spend more than you want to.

This could be a temporary workaround, not ideal for larger scale automated testing. Running emulator locally will be much time-efficient to run the test cases.

[itsravenous]

This could be a temporary workaround, not ideal for larger scale automated testing. Running emulator locally will be much time-efficient to run the test cases.

Totally agree. For me, it works OK as my project is fairly small. But of course, if we could emulate storage then that would be much, much better.

[Canavell]

We need this! It's even looks strange when you have emulators on all your products except storage
I absolutlty love your solution for emulators, but add storage emulator please

[gpfister]

This is also a big requirement on our side, Storage plays a big role in our app. Please add Storage as soon as possible.

[mrudling]

Please add the storage emulator, really need this.

[therealmodbom]

a big +1 for me too

[pbilka46]

👍 ! I would also like to have storage emulator

[abeisgoat]

Hey folks, if you're following this thread and willing to be an Alpha tester of "Bug fixes and Performance Improvements 😉" related to this bug, follow these steps:

1. Sign up for the Firebase Alpha program
2. Send an email to abehaskins@google.com, make sure to send it from the email address you will use for testing.

Please only do this if you're seriously interested in trying out an early-stage, buggy "performance improvement" and providing feedback! There will be issues and some parts of the setup will be hard, but we want to know what you think.

[eru123]

Is there any update about this feature request? badly needed this to test file upload features of my project.

[warting]

Is there any update about this feature request? badly needed this to test file upload features of my project.

Looks like it is just around the corner :)
Check the commits included in v9.8.0

[oittaa]

customTime seems to be missing from stored metadata https://github.com/firebase/firebase-tools/blob/93abee929d79b2216a31af98871b6d29588810be/src/emulator/storage/metadata.ts

https://cloud.google.com/storage/docs/metadata#custom-time

For example here's how the Python client uses customTime:

https://github.com/googleapis/python-storage/blob/440a0a4ffe00b1f7c562b0e9c1e47dbadeca33e1/google/cloud/storage/blob.py#L102

[samtstern]

@warting follow the instructions that @abeisgoat posted here if you're interested:
#1738 (comment)

@oittaa let's not use this issue for feedback, there are too many people subscribed here and adding comments will create a lot of noise. You can contact @abeisgoat with feedback using the instructions above or if that is not working for some reason you can file a new issue on this repo.

[eduardolat]

+1 it's really important to us to have a COMPLETE emulator suite wrapping all the firebase services. We need to make local tests and integrate with CI/CD but if one is missing we can't build good tests around our entire application.

[gursesl]

Hoping for an announcement on Google I/O ;)

[mono0926]

I’ve found the Google I/O session🐶

Local development using the new Storage Emulator: https://events.google.com/io/session/21e0d8b3-cf63-4931-a6d7-ab8a8e2b10c9

[abeisgoat]

@mono0926 shhhh 😉

[abeisgoat]

Hey folks, at Google I/O we'll be announcing that the Cloud Storage for Firebase emulator, released in firebase-tools@9.11.0, is available in beta today. Thank you for your patience and support.

As someone who has worked on Firebase for nearly a decade it's thrilling to see the excitement of the community and to be able to keep delivering features that we only dreamed of back when we were a 10 person startup working from an apartment. We always wanted to offer a way for folks to build with Firebase entirely offline and with the Cloud Storage emulator we're finally filling in the last major gap for the Emulator Suite. It's been a long time coming!

Hope you enjoy, excited to see all the bugs you report,
abe