Skip to content

nextjs-end rollout failed #302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
v0lsh3bnik opened this issue Jul 29, 2024 · 34 comments
Open

nextjs-end rollout failed #302

v0lsh3bnik opened this issue Jul 29, 2024 · 34 comments

Comments

@v0lsh3bnik
Copy link

I am using the nextjs-end repo for firebase app hosting, the build is working fine but i get an error on the rollout. I looked through the logs but nothing. Can someone help me please ?

@WCressman
Copy link

WCressman commented Aug 1, 2024

I currently have the same issue, I am following the tutorial step by step and App Hosting is failing
image

@ni7r0x
Copy link

ni7r0x commented Aug 2, 2024

In my case , I found out the issue by looking in Google Cloud > Logging > Logs explorer > SEVERITY > Error :

google.cloud.run.v2.Services.CreateService returns this error :

template.containers.resources.limits.cpu: Invalid value specified for cpu.
For the specified value, maxScale may not exceed 10.
...

Solved by adding the following lines to apphosting.yaml :

# Settings for Cloud Run
runConfig:
  minInstances: 0
  maxInstances: 10
  concurrency: 50
  cpu: 1
  memoryMiB: 512

I guess the Rollout Failure should be more verbose than that.

@AntonioGally
Copy link

I had the same issue with my nexjs deployments:
Google Cloud Build was successfully building my app, but in firebase console, my deployment launch was failing.
I've added this configuration in my apphosting.yaml that @ni7r0x commented (tks) and my deployment was completed

It would be great if the official docs mentioned this apphosting.yaml configuration file in the Basic Deployment section, and tell us if this file is required.

image image

@gyula-szucs
Copy link

Had the same mildly frustrating issue.
Thank you @ni7r0x.

@r-dev-limited
Copy link

Mine just says all of a sudden: Ready condition status changed to False for Revision backlink-space-build-2024-10-28-003 with message: Deadline exceeded

@RebinX1
Copy link

RebinX1 commented Nov 19, 2024

i also have the same issue now, i have added the apphosting.yaml in my project but still i get the same error, one more thing is that at the start i was able to deploy 2 build to apphosting but after working on my app for a bit i started to get this error.

Screenshot 2024-11-19 at 6 37 06 PM

@ItsTeccs
Copy link

Currently experiencing the same issue with a valid apphosting.yaml file present in my project. Google console shows in the logs that the build was successful in the cloud, and my build locally is fine, but the rollout fails every time.

@vojdan
Copy link

vojdan commented Jan 28, 2025

I'm suddenly getting failed deploys with a successful build as well, just like in @RebinX1 screenshot.

Had to delete and redeploy the backend to get a working deploy.
Still seeing "Resource 'my-backend' already exists." errors in the GCP logs on re-deploy.

@josephkabba
Copy link

josephkabba commented Feb 4, 2025

Having the same issue, it looks like next build is not being run, and the official docs don't explain how to get the environment to build next-js apps for some reason, It thinks it's an angular application, which is not the case. Please, the Firebase team, make this very clear in the documentation.

@ItsBenDov
Copy link

ItsBenDov commented Apr 17, 2025

I have the same problem too. it looks like the build was completed, and the revisions in Cloud Run show that the build was successful, but firebase still shows that the build have failed in the console.

@Msekoo
Copy link

Msekoo commented Apr 18, 2025

same problem, and i fixed apphosting.yaml config problem, the revisions in Cloud Run show that the build was successful, no any error. but firebase still shows that the build have failed in the console.

@ItsBenDov
Copy link

Looks like that's happening for a lot of users
see this reddit thread:
https://www.reddit.com/r/Firebase/comments/1k1f321/error_occurred_in_app_hosting_rollout/

report it to the firebase team for them to fix it.

@jimmy-chiang
Copy link

jimmy-chiang commented Apr 18, 2025

Same here. The build was successful but the rollout failed. And the logs explorer says:
"The request was invalid: build "build-2025-04-18-002" was not found and is invalid"

Image

@ItsBenDov
Copy link

This is their response:

"The App Hosting expert team has now identified the cause. They found that there’s a brief period of time between when you create a build and when it’s safe to start the rollout. In the cases they analyzed, the build and the rollout were triggered in the same second most likely through a script.
As a temporary workaround, they recommend waiting a couple of seconds between triggering the build and the rollout. The team is already working on a fix to handle this better in the future."

But I simply don't understand what I should do? like how can I make it wait a couple of seconds?

@ItsTeccs
Copy link

I'm back to this thread because I was able to get successful build(s) by deleting the container in cloud run then re-building. That was, until last night & this morning. No matter what my local build is good, my cloud build is good, but my rollout fails. This is a critical level issue in my own opinion given the amount of users experiencing problems and the fact that the platform is wholly unusable if we cannot get a successful rollout or meaningful logs to tell us why we are not getting successful rollouts. I've lost two weekends of development time in the last few months because of this issue.

@ItsTeccs
Copy link

@ItsBenDov When you receive a response from Firebase in context of what to do, could you please post here again as well?

What category did you file this under when speaking to Firebase support? I will likely submit a ticket too but it would make sense for those of us experiencing issues to attempt to categorically pool the support tickets to make it clearer to Firebase that this is a serious issue affecting many users.

@ItsBenDov
Copy link

I reported it here,
and marked the urgency as: "This is preventing me from using Firebase. There is no workaround." under "App hosting" and "NodeJS". The support team responded to the ticket quickly, but I think that as soon as more people will report this very critical issue, they will try to address it asap.

@rzfang
Copy link

rzfang commented Apr 21, 2025

read the document and tried following config in apphosting.yaml, but got no luck 😞

# try workaround to escape the rollout problem from Firebase.
scripts:
  buildCommand: next build --no-lint && sleep 5
  runCommand: node dist/index.js && sleep 5

@Msekoo
Copy link

Msekoo commented Apr 21, 2025

Here’s how I resolved this issue:

Solution 1: Delete and recreate the Firebase App Hosting backend

The simplest fix in many cases is to delete your existing Firebase App Hosting backend and create a new one from scratch. This worked reliably across multiple projects for me.

Solution 2: Manually edit the Cloud Build history 1. Go to Google Cloud Console > Cloud Build > Build History. 2. Click on the YAML tab for a recent build. 3. Edit the file and remove any entries that reference build dates not associated with your listed builds. 4. Go back to Firebase, and manually trigger a rollout of your most recent deployment to GitHub.

Note: This approach is not recommended, and you’ll have to repeat it every time you deploy. I only use it on one of my apps where downtime isn’t an option.

Important note for users with GoDaddy DNS:

If you’re using GoDaddy’s nameservers, be cautious. After deleting and recreating a Firebase backend, I experienced issues where Google Cloud was effectively blocked by GoDaddy’s internal systems. It took multiple escalations for their support to acknowledge the issue. I strongly recommend switching to Google Cloud DNS if you plan to delete and recreate your Firebase project that had a custom domain to avoid such delays.

How to avoid this issue in the first place:

In my experience, this problem often arises when you push to GitHub and deploy back-to-back too quickly—like realizing you missed something immediately after deploying and pushing a second fix right away. Avoid making consecutive rapid deployments. Let the first deployment fully complete before initiating another.

This is not helpful, you may have another problem

@julian-printemps
Copy link

According to Firebase support, it seems like it is happening when we attempt to rollout shortly after creating a backend. They're currently investigating to solve this issue.

@ItsTeccs
Copy link

According to Firebase support, it seems like it is happening when we attempt to rollout shortly after creating a backend. They're currently investigating to solve this issue.

Image

I created a new backend to test this theory out and waited about 10 minutes before deploying for the first time; I disabled auto-rollout and clicked "finish" rather than "finish and deploy" to let the backend sit idle before deploying initially.

This is the same project, same branch, same commit, same configuration on two different backends. One fails but the other succeeds.

@Msekoo
Copy link

Msekoo commented Apr 22, 2025

According to Firebase support, it seems like it is happening when we attempt to rollout shortly after creating a backend. They're currently investigating to solve this issue.

My backend has been created and used for a long time, but I have not been able to publish it successfully since recently.

@keiziiisan
Copy link

An incident has been published on the Firebase Status Dashboard.

https://status.firebase.google.com/incidents/HkK8snnXw4jYrUVfYbNw

@ItsBenDov
Copy link

The same, also creating a new backend doesn't solve the problem for me. It's crazy that there is such a major disruption that prevents making new rollouts and practically paralyzing my app for over a week.

@Hiraqui
Copy link

Hiraqui commented Apr 24, 2025

Hi I'm having the same issue and no error in the logs

@BlaShadow
Copy link

I'm having the same issue, creating a new backend with demo project still not working.

@itomin
Copy link

itomin commented Apr 24, 2025

We are having the same issue. The backend was already deployed and running, we just wanted to promote a very simple change of code, build is green, rollout is not possible. No logging, no details, wasting hours to understand what is going on.

@ItsTeccs
Copy link

ItsTeccs commented Apr 25, 2025

The incident report claims its fixed but its not.

I am still getting failed rollouts across all projects & backends. Here is a rollout history from one of my backends proving it's the same commit and branch failing rollouts.

Image

I deleted the underlying container before re-deploying so it could re-build and rollout again. Before this recent incident I did this to fix failed rollouts and it worked perfectly fine before it started failing every time last week.

@jimmy-chiang
Copy link

jimmy-chiang commented Apr 25, 2025

Just got a reply from the support team.

They said:

We’ve identified that the changes are not being fully propagated in some regions, which is why you’re still having the issue.

I'm using asia-east and can identify the rollout is still broken.

@julian-printemps
Copy link

For some mysterious reason, it worked on a brand new backend with auto-deploy from GitHub enabled. I didn't deploy anything when creating the backend, but auto-deploy was set to true. I haven't encountered any deployment failures so far. Could it be related to manual deployments?

@BAYiTUPAi
Copy link

An incident has been published on the Firebase Status Dashboard.

https://status.firebase.google.com/incidents/HkK8snnXw4jYrUVfYbNw

The status update says fixed but it's still broken for me. In us-central1.

@ItsTeccs
Copy link

For some mysterious reason, it worked on a brand new backend with auto-deploy from GitHub enabled. I didn't deploy anything when creating the backend, but auto-deploy was set to true. I haven't encountered any deployment failures so far. Could it be related to manual deployments?

I just tried enabling automatic rollout on two different projects and adding some whitespace to a file to generate a commit that would trigger the auto-rollout and both backends built first try. Manual rollouts are absolutely part of the issue to why the rollouts are not succeeding. Fantastic find, thank you so much.

@jhk115
Copy link

jhk115 commented Apr 25, 2025

Same for me. Auto-rollout works but not through "create rollout". Interestingly, the "save and deploy" button in the auto deployment settings page worked too.

@BlaShadow
Copy link

For some mysterious reason, it worked on a brand new backend with auto-deploy from GitHub enabled. I didn't deploy anything when creating the backend, but auto-deploy was set to true. I haven't encountered any deployment failures so far. Could it be related to manual deployments?

This workaround worked for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests