New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List and delete deployments #215

Open
Nowaker opened this Issue Sep 2, 2016 · 49 comments

Comments

Projects
None yet
@Nowaker
Copy link

Nowaker commented Sep 2, 2016

I've just noticed by Firebase hosting usage is now almost 1 GB. Quite a lot given the fact our website is only 20 MB.

nowaker@nwkr-desktop ~/projekty/virtkick/website (git)-[master] % du -hs build 
20M     build

It seems all previous deployments are kept by Firebase and they're visible at https://console.firebase.google.com/project/PROJECTNAME/hosting/main.

Deleting 90 deployments one by one manually is out of question. There's a big need to have a way to list deployments via CLI and delete them.

@brendanlim

This comment has been minimized.

Copy link
Contributor

brendanlim commented Sep 30, 2016

@Nowaker This is definitely on our radar and something we're looking to improve

@zhenhaolei

This comment has been minimized.

Copy link

zhenhaolei commented Apr 12, 2017

hey @brendanlim any updates here? We are also seeing infinite loading in the hosting module, looks like we have too many deployments :(

Thanks!

Error: too_big: The data requested exceeds the maximum size that can be accessed with a single request. at r (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8597) at H (third_party/javascript/firebase/firebase_js_minified.jslib:126) at Object.eval [as H] (third_party/javascript/firebase/firebase_js_minified.jslib:206) at eval (third_party/javascript/firebase/firebase_js_minified.jslib:190) at Kh.g.Id (third_party/javascript/firebase/firebase_js_minified.jslib:196) at yh.Id (third_party/javascript/firebase/firebase_js_minified.jslib:186) at qh.eval [as zg] (third_party/javascript/firebase/firebase_js_minified.jslib:184) at th (third_party/javascript/firebase/firebase_js_minified.jslib:178) at WebSocket.ua.onmessage (third_party/javascript/firebase/firebase_js_minified.jslib:177) at WebSocket.b [as __zone_symbol___onmessage] (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8568) at w.invokeTask (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8611) at u.runTask (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8601) at WebSocket.invoke (rs=AON9PunUVzISsT9OTTHxul9qDyFtbYjNGA:8613)

@Nowaker

This comment has been minimized.

Copy link

Nowaker commented Jul 9, 2017

@brendanlim @mbleigh Any update on this? Our deployment history is constantly growing, each CI build at a time, and it's beyond our human capabilities to keep up with deleting them!

@laurenzlong laurenzlong assigned mbleigh and unassigned brendanlim Jul 10, 2017

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jul 10, 2017

We're aware of the issue here and are considering the best way to tackle it. In general we don't want you to feel worried about having a growing history of versions. Curious for anyone who comes across this (vote with emoji on this comment): which would you like best?

  • 🎉 The ability to list and possibly batch delete old versions
  • 👍 Old versions are only kept for a certain amount of time unless "pinned" in some way
  • ❤️ Only a certain number of old versions are kept unless "pinned" in some way
@Nowaker

This comment has been minimized.

Copy link

Nowaker commented Jul 11, 2017

"The ability to list and possibly batch delete old versions" is a primitive. APIs are built using primitives. These primitives allow users to achieve whatever they want without bugging anyone (in this case, you @mbleigh).

The other two are high level features. They're cool. But then comes a question: how do I pin or unpin the version via API or firebase-tools? As you can imagine, once again someone will have to click through the list in Firebase UI one by one, just like to delete old deployments today. 🤣

To sum up, high level features are great and needed, but CRUD primitives in API are super important for tools like Google Cloud Platform or Firebase.

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jul 11, 2017

@Nowaker I don't disagree, and both are important. It's fair to say it's a false dichotomy between the first option and the other two. FWIW, in order to implement the second two options we'd pretty much have to implement everything necessary for the first anyway.

This stuff is definitely on our radar, but I don't have specifics on when we'll have something to show.

@CaptainCodeman

This comment has been minimized.

Copy link

CaptainCodeman commented Jul 18, 2017

I would make do with a checkbox to select all / multiple entries on a page and delete them in one go. Also, make it possible to dismiss the confirmation dialog by pressing enter to confirm.

@patrickmcd

This comment has been minimized.

Copy link

patrickmcd commented Jul 18, 2017

You could also not hide the delete functionality. Right now you:

  1. Rollover deployment row.
  2. Click the three dots menu.
  3. Click delete to call up the confirmation.
  4. Click delete again.

If each row had it's own delete button, and if you could hold shift to bypass the confirmation, you could fly though them.

@titpetric

This comment has been minimized.

Copy link

titpetric commented Aug 11, 2017

Is an REST API for listing and deleting deployments available, if one wanted to implement this functionality on their own (or even implement it in firebase-tools, and submit a PR)? If the endpoints are just not there, I can see how that might be a blocking issue. Is this on any kind of road map with something similar to a due date?

@Thkasis

This comment has been minimized.

Copy link

Thkasis commented Aug 17, 2017

Any news on this? It would definitely help to have a simple way of removing previous deployments in CLI. Thanks for the good work.

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Aug 17, 2017

No news on this yet, but it's an area of active interest for the team. Thanks for your patience!

@shaun-sweet

This comment has been minimized.

Copy link

shaun-sweet commented Oct 2, 2017

Any news on this? We have so many we have to manually remove, one by one.

@glennlawyer

This comment has been minimized.

Copy link

glennlawyer commented Oct 20, 2017

Would also really appreciate some progress on this!

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Oct 20, 2017

We're doing a lot of work on our deployment infrastructure right now that will take a bit of time to come to fruition. This issue is definitely on our mind as we do that work.

@titpetric

This comment has been minimized.

Copy link

titpetric commented Oct 25, 2017

@mbleigh hey how about locking this topic. I'd like to be notified when it's done, but I really don't want to read all the "me too" comments.

@ludvig-larsson

This comment has been minimized.

Copy link

ludvig-larsson commented Nov 8, 2017

"Only a certain number of old versions are kept unless "pinned" in some way" is exactly what we need.

guoguo12 added a commit to guoguo12/hivemind that referenced this issue Dec 13, 2017

Stop using Firebase
I'm well over my Firebase Storage hosting limit. Not because I actually
store a lot of data, but because there's no way to delete past versions
of files. See firebase/firebase-tools#215.
@rtm

This comment has been minimized.

Copy link

rtm commented Jan 25, 2018

The other problem I have is that I when I look at this list of deployments, I have no idea which is which. I version my app in a couple of ways, including the version in package.json, and an notion of "builds", so I could also provide a version and/or build # to firebase deploy, and then if that could be listed in the list of deployed versions it would be fantastic. As it stands, I see 100 deployed versions and have no idea which is which.

@a-xin

This comment has been minimized.

Copy link
Contributor

a-xin commented Jan 25, 2018

@rtm You can specify a message for deploys which is shown in the list of deployments:

firebase deploy --message "build 1234"

The option is not listed on the documentation pages, but it's listed when running:
firebase help deploy

@rtm

This comment has been minimized.

Copy link

rtm commented Jan 26, 2018

@a-xin Thank you. I had totally missed that and it's very useful!

@haraldreingruber

This comment has been minimized.

Copy link

haraldreingruber commented Jan 26, 2018

It would be great if the priority of this issue could be increased, as we do continuous deployment of every git commit, and each deployment is a couple of hundred MB. Would be nice if we were able to integrate some commands for cleaning up old deployments in the CD script as well, in order to reduce the overall storage consumption.

@dinvlad

This comment has been minimized.

Copy link

dinvlad commented Feb 25, 2018

As a workaround, would it be possible to reuse the files whose filename and contents haven't changed (according to some hash)?

E.g. if Webpack generates chunks with stable hashes (e.g. employing HashedModuleIdsPlugin), do these files get uploaded on every deploy (even though they truly haven't changed)?

That could significantly cut down on the amount of file updates on each deployment (down to kbytes of our application code, if vendor libs haven't changed).

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Feb 26, 2018

@sejr

This comment has been minimized.

Copy link

sejr commented May 15, 2018

I am glad to see that this is a priority. I am on the Spark plan and have a web application that is somewhere around 20MB. I have been a little liberal with my deployments so I have 300 to go through and delete. 😅

In addition to the limited # of deployments in history, I also like @dinvlad's idea of reusing files.

@sboyd

This comment has been minimized.

Copy link

sboyd commented May 15, 2018

@sejr one thing I do that helps is use the keyboard as much as possible.
I click the elipses, hit the down arrow then enter, then the tab button 3 times then enter
And repeat.....hopefully that helps.

@billiaug

This comment has been minimized.

Copy link

billiaug commented Jun 1, 2018

@mbleigh Any news or progression on this?

@r1th4l1n

This comment has been minimized.

Copy link

r1th4l1n commented Jun 1, 2018

@mbleigh can you give us a status?

@haraldreingruber

This comment has been minimized.

Copy link

haraldreingruber commented Jun 27, 2018

@mbleigh Looking forward to more transparency. Maybe you could elaborate more about the general firebase-tools roadmap, and the plans regarding this issue?
(It's the highest voted issue btw, almost double the votes compared the second highest one).

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jun 29, 2018

Sorry for the long delay on addressing this -- it's high on our list, but we've been heavily investing in an infrastructure project that sets the foundation to take on many smaller projects like this one.

I can't promise exact timelines, but this definitely hasn't been forgotten.

@Nowaker

This comment has been minimized.

Copy link

Nowaker commented Jun 29, 2018

Maybe not forgotten but surely ignored. A deployment can be deleted in the UI with a couple clicks - there's no reason why the same couldn't be easily exposed via API. I'm extremely disappointed by Firebase product roadmap, as well as the fate of Divshot.

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jun 29, 2018

@rtm

This comment has been minimized.

Copy link

rtm commented Jun 30, 2018

@mbleigh

but there are large changes necessary to do what we want to do.

Maybe the seething masses would be pacified by some suitably vague hand-waving about what these large change are, and what other earth-shattering features they would enable. Failing that, it's not unreasonable for some folks to get their undies in a twist about what does seem like merely ignoring the feature, since it's going on almost two years now.

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jun 30, 2018

It's pretty straightforward -- Firebase Hosting has never had a public API, and the current method for listing and deleting deployments in the console is not well-suited to publicizing. Since the CLI is open-source, we need to make sure any actions exposed through it are appropriately scalable.

We're in the process of migrating a large portion of Firebase Hosting onto Google infrastructure that will make it more scalable and also provide a strong foundation for properly having APIs for things. This effort is very good for the long-term health of Firebase Hosting but has been a substantial investment for the team that has resulted in a long period of no apparent progress.

I appreciate that many of you feel strongly about this -- if you reach out to support, we're happy to manually prune your older deploys in the interim. We hope the wait won't be much longer, but as with any software, picking exact dates for things is usually a bad idea.

I hope this clears things up a bit, and thanks to all Firebase Hosting users for bearing with us 😄

@haraldreingruber

This comment has been minimized.

Copy link

haraldreingruber commented Jul 3, 2018

@mbleigh Thanks a lot for the heads up.

@rudresh4

This comment has been minimized.

Copy link

rudresh4 commented Jul 18, 2018

@mbleigh I'm not sure why you need to make this task dependent on a public API or any infrastructure change.

This can be solved in the short term with a button in the Firebase console "Prune old deployments".

Since the console is already making calls to the backend, it can get a list of all deployments and then call delete. I'm assuming the API on the backend doesn't take a batch list of deployments, so this button might cause some issues if you do it synchronously with one deployment at a time.

Anyways, the point is that don't over engineer a solution when all you need to do is add a button to an already established API in your backend. Especially, don't make your users wait 2 years for it.

Not professional at all.

@HenrikBechmann

This comment has been minimized.

Copy link

HenrikBechmann commented Jul 21, 2018

I don't know what everyone is grousing about. It's RELAXING to delete old deployments one-by-one. I just cleared a backlog of 72. Then there was double checking to make sure I'd gotten them all (I missed one) -- again very satisfying. I felt virtuous, and dare I say SUPERIOR. Taking care of details; due diligence. Administrative fastidiousness! My only regret was that I didn't have MORE to clear up. I actually had to play solitaire for a WHOLE HOUR afterwards, just to feel well enough to move to my next task.

And two years (and counting) to get this feature?!? Pfftt. Everyone knows that engineers have to take CARE to do these things right! And it's a huge BUDGET that's required. I mean think of all the effort WAYMO and the rest of Alphabet is taking. Google couldn't POSSIBLY afford to put more resources into this feature.

And besides, we're just DEVELOPERS! DOUBLE Pfftt. How do you expect Google to survive by putting DEVELOPERS at the head of the queue?!? I mean really!

No, for the good of us all, and particularly for our well-being, I personally vote that the dev team here continue to work on this for at LEAST another two years before implementing this feature. For God's sake be careful! Don't be rash; don't be rushed!

And for heaven's sake, don't do anything impulsive like having a temp feature of a quick delete button or some such; that would just divert resources from much more important things.

All good as is; please don't make ANY changes to your process, such as, say MANAGEMENT changes!

Oops, gotta go. Time for my nap (I'm exhausted from all the concentration required to delete all those deployments). And besides with such a productive time, I certainly deserve some time off!

@sanjeevhalyal

This comment has been minimized.

Copy link

sanjeevhalyal commented Jul 22, 2018

I have 125 of them to delete.

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jul 23, 2018

🤔 Somehow I feel like there's a bit of sarcasm happening in this thread... 🙃

Folks, I really do get the frustration. To reiterate a statement from my previous comment: if you reach out to support, we're happy to manually prune your older deploys on your behalf in the interim. Just let us know how many old versions you would like to keep around.

As for why we're not releasing an interim band-aid solution -- well, the support intervention is our band-aid. We'd prefer not to burn limited engineering resources on a system we're actively working to replace, which is why this ends up in a bit of a tough spot.

Hang In There, Baby

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Aug 25, 2018

Oh look, someone seems to have dropped a gist that might be relevant to this thread...

Ducks Out

P.S. We're still working on a better, automated solution but in the meantime this is a much better band-aid. 😼

@haraldreingruber

This comment has been minimized.

Copy link

haraldreingruber commented Aug 25, 2018

Thanks for taking the time to provide a work-around.

@anatesan-stream

This comment has been minimized.

Copy link

anatesan-stream commented Oct 6, 2018

The script in the gist above works fine in terms of deleting the versions, but I don't see my Storage Usage come down (does it take a while to recompute this after the deletes of the versions?)

@zevdg

This comment has been minimized.

Copy link

zevdg commented Oct 6, 2018

does it take a while to recompute this after the deletes of the versions?

In my experience, yes. Even if you use the gui to manually delete a bunch of versions, it takes some time for the usage numbers to change.

@wahidyankf

This comment has been minimized.

Copy link

wahidyankf commented Oct 29, 2018

Really thanks @mbleigh for providing us with a much better band-aid. 🙏

@alexanderwhatley

This comment has been minimized.

Copy link

alexanderwhatley commented Dec 16, 2018

Any updates @mbleigh?

@jackcw

This comment has been minimized.

Copy link

jackcw commented Dec 17, 2018

Have you looked at the hosting rest api @alexanderwhatley its not a ui solution but the new Hosting Rest API should enable you to do everything you need to trim up your deployments

Hosting REST API

@natedalliard

This comment has been minimized.

Copy link

natedalliard commented Dec 17, 2018

Have you looked into it @jackcw ? Cause I can only find create and list methods. No delete method.

@jackcw

This comment has been minimized.

Copy link

jackcw commented Dec 17, 2018

I havent actually used it but as far as i can see there is a delete on the versions endpoint and the list on releases includes the version object so i assume you do a list of releases and take the version id and hit the version delete endpoints

@natedalliard

This comment has been minimized.

Copy link

natedalliard commented Dec 17, 2018

Sorry my bad. I misunderstood the meaning of versions and releases. You can now delete older releases which are called versions with the API.

@andipaetzold

This comment has been minimized.

Copy link

andipaetzold commented Dec 17, 2018

I created a small shell script that is executed once a day with a cron job to remove all old releases.
Here is the gist. It requires jq to parse the JSON. It basically does what @jackcw wrote: It iterates over all releases and deletes them.

I am not an expert in writing shell scripts or jq - but the result is what counts, I guess. The script works quite reliably for me. Feel free to use it yourself.

@mbleigh

This comment has been minimized.

Copy link
Contributor

mbleigh commented Jan 9, 2019

Internal tracking id: 113235359

@mbleigh mbleigh added the hosting label Jan 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment