Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Waiting for health checks... running forever. #2456

Open
Shyrogan opened this issue Jun 25, 2024 · 15 comments
Open

Waiting for health checks... running forever. #2456

Shyrogan opened this issue Jun 25, 2024 · 15 comments

Comments

@Shyrogan
Copy link

Describe the bug
Recently I have been unable to launch my supabase local project using the supabase start command.
The command remains stuck on Waiting for health check and never runs succesfully.

  1. Studio is available normally.
  2. Authentication or any service related to the database seem to work.

System information
Rerun the failing command with --create-ticket flag.

  • Ticket ID: d888349bfa5f409fb91e7b4eb4f71d98
  • Version of OS: macOS d888349bfa5f409fb91e7b4eb4f71d98
  • Version of CLI: 1.178.2
  • Version of Docker: Docker version 26.1.4, build 5650f9b102
  • Versions of services:
        SERVICE IMAGE      │      LOCAL       │ LINKED
  ─────────────────────────┼──────────────────┼─────────
    supabase/postgres      │ 15.1.1.61        │ -
    supabase/gotrue        │ v2.151.0         │ -
    postgrest/postgrest    │ v12.0.1          │ -
    supabase/realtime      │ v2.28.32         │ -
    supabase/storage-api   │ v1.0.6           │ -
    supabase/edge-runtime  │ v1.54.6          │ -
    supabase/studio        │ 20240422-5cf8f30 │ -
    supabase/postgres-meta │ v0.80.0          │ -
    supabase/logflare      │ 1.4.0            │ -
    supabase/supavisor     │ 1.1.56           │ -
    darthsim/imgproxy      │ v3.8.0           │ -

✨  Done in 0.19s.
@sweatybridge
Copy link
Contributor

Could you report which services are unhealthy? ie. when cli says Waiting for health check..., run the following command in another terminal.

docker ps -a

@Shyrogan
Copy link
Author 

CONTAINER ID   IMAGE                                             COMMAND                  CREATED              STATUS                        PORTS                                                                  NAMES
f6e2d0a98988   public.ecr.aws/supabase/studio:20240422-5cf8f30   "docker-entrypoint.s…"   45 seconds ago       Up 43 seconds (healthy)       0.0.0.0:54323->3000/tcp, :::54323->3000/tcp                            supabase_studio_mentor
12431e0a3e21   public.ecr.aws/supabase/postgres-meta:v0.80.0     "docker-entrypoint.s…"   45 seconds ago       Up 44 seconds (healthy)       8080/tcp                                                               supabase_pg_meta_mentor
818fcf85ea2e   public.ecr.aws/supabase/edge-runtime:v1.54.6      "sh -c 'cat <<'EOF' …"   45 seconds ago       Up 44 seconds                 0.0.0.0:32768->8081/tcp, :::32768->8081/tcp                            supabase_edge_runtime_mentor
2d16a4fadf28   public.ecr.aws/supabase/imgproxy:v3.8.0           "imgproxy"               46 seconds ago       Up 44 seconds (healthy)       8080/tcp                                                               supabase_imgproxy_mentor
1fc177aa7072   public.ecr.aws/supabase/storage-api:v1.0.6        "docker-entrypoint.s…"   46 seconds ago       Up 45 seconds (healthy)       5000/tcp                                                               supabase_storage_mentor
48ddd978df6c   public.ecr.aws/supabase/postgrest:v12.0.1         "postgrest"              46 seconds ago       Up 45 seconds                 3000/tcp                                                               supabase_rest_mentor
9461c1d93559   public.ecr.aws/supabase/realtime:v2.28.32         "/usr/bin/tini -s -g…"   47 seconds ago       Up 46 seconds (healthy)       4000/tcp                                                               supabase_realtime_mentor
4df7ed5dc2a5   public.ecr.aws/supabase/inbucket:3.0.3            "/start-inbucket.sh …"   47 seconds ago       Up 46 seconds (healthy)       1100/tcp, 2500/tcp, 0.0.0.0:54324->9000/tcp, :::54324->9000/tcp        supabase_inbucket_mentor
02a8792d2d43   public.ecr.aws/supabase/gotrue:v2.151.0           "auth"                   47 seconds ago       Up 46 seconds (healthy)       9999/tcp                                                               supabase_auth_mentor
c8e38227fed7   public.ecr.aws/supabase/kong:2.8.1                "sh -c 'cat <<'EOF' …"   47 seconds ago       Up 46 seconds (healthy)       8001/tcp, 8443-8444/tcp, 0.0.0.0:54321->8000/tcp, :::54321->8000/tcp   supabase_kong_mentor
d7ab72e1bbfc   public.ecr.aws/supabase/postgres:15.1.1.61        "sh -c 'cat <<'EOF' …"   About a minute ago   Up About a minute (healthy)   0.0.0.0:54322->5432/tcp, :::54322->5432/tcp                            supabase_db_mentor

@Shyrogan
Copy link
Author

Here are the logs for postgrest:

27/Jun/2024:09:41:01 +0000: Starting PostgREST 12.0.1 (cd38da5)...
27/Jun/2024:09:41:01 +0000: Attempting to connect to the database...
27/Jun/2024:09:41:01 +0000: Connection successful
27/Jun/2024:09:41:01 +0000: Admin server listening on port 3001
27/Jun/2024:09:41:01 +0000: Listening on port 3000
27/Jun/2024:09:41:01 +0000: Listening for notifications on the pgrst channel
27/Jun/2024:09:41:01 +0000: Config reloaded
27/Jun/2024:09:41:01 +0000: Schema cache loaded

and edge-runtime:

Download https://deno.land/std/http/status.ts
Download https://deno.land/x/jose@v4.13.1/index.ts
Implicitly using latest version (0.224.0) for https://deno.land/std/http/status.ts
Download https://deno.land/std@0.224.0/http/status.ts
Download https://deno.land/x/jose@v4.13.1/jwe/compact/decrypt.ts
Download https://deno.land/x/jose@v4.13.1/jwe/flattened/decrypt.ts
Download https://deno.land/x/jose@v4.13.1/jwe/general/decrypt.ts
Download https://deno.land/x/jose@v4.13.1/jwe/general/encrypt.ts
Download https://deno.land/x/jose@v4.13.1/jws/compact/verify.ts
Download https://deno.land/x/jose@v4.13.1/jws/flattened/verify.ts
Download https://deno.land/x/jose@v4.13.1/jws/general/verify.ts
Download https://deno.land/x/jose@v4.13.1/jwt/verify.ts
Download https://deno.land/x/jose@v4.13.1/jwt/decrypt.ts
Download https://deno.land/x/jose@v4.13.1/jwe/compact/encrypt.ts
Download https://deno.land/x/jose@v4.13.1/jwe/flattened/encrypt.ts
Download https://deno.land/x/jose@v4.13.1/jws/compact/sign.ts
Download https://deno.land/x/jose@v4.13.1/jws/flattened/sign.ts
Download https://deno.land/x/jose@v4.13.1/jws/general/sign.ts
Download https://deno.land/x/jose@v4.13.1/jwt/sign.ts
Download https://deno.land/x/jose@v4.13.1/jwt/encrypt.ts
Download https://deno.land/x/jose@v4.13.1/jwk/thumbprint.ts
Download https://deno.land/x/jose@v4.13.1/jwk/embedded.ts
Download https://deno.land/x/jose@v4.13.1/jwks/local.ts
Download https://deno.land/x/jose@v4.13.1/jwks/remote.ts
Download https://deno.land/x/jose@v4.13.1/jwt/unsecured.ts
Download https://deno.land/x/jose@v4.13.1/key/export.ts
Download https://deno.land/x/jose@v4.13.1/key/import.ts
Download https://deno.land/x/jose@v4.13.1/util/decode_protected_header.ts
Download https://deno.land/x/jose@v4.13.1/util/decode_jwt.ts
Download https://deno.land/x/jose@v4.13.1/util/errors.ts
Download https://deno.land/x/jose@v4.13.1/key/generate_key_pair.ts
Download https://deno.land/x/jose@v4.13.1/key/generate_secret.ts
Download https://deno.land/x/jose@v4.13.1/util/base64url.ts
Download https://deno.land/x/jose@v4.13.1/lib/buffer_utils.ts
Download https://deno.land/x/jose@v4.13.1/runtime/base64url.ts
Download https://deno.land/x/jose@v4.13.1/runtime/decrypt.ts
Download https://deno.land/x/jose@v4.13.1/runtime/zlib.ts
Download https://deno.land/x/jose@v4.13.1/lib/is_disjoint.ts
Download https://deno.land/x/jose@v4.13.1/lib/is_object.ts
Download https://deno.land/x/jose@v4.13.1/lib/decrypt_key_management.ts
Download https://deno.land/x/jose@v4.13.1/lib/cek.ts
Download https://deno.land/x/jose@v4.13.1/lib/validate_crit.ts
Download https://deno.land/x/jose@v4.13.1/lib/validate_algorithms.ts
Download https://deno.land/x/jose@v4.13.1/lib/encrypt_key_management.ts
Download https://deno.land/x/jose@v4.13.1/runtime/verify.ts
Download https://deno.land/x/jose@v4.13.1/lib/check_key_type.ts
Download https://deno.land/x/jose@v4.13.1/lib/jwt_claims_set.ts
Download https://deno.land/x/jose@v4.13.1/runtime/encrypt.ts
Download https://deno.land/x/jose@v4.13.1/lib/iv.ts
Download https://deno.land/x/jose@v4.13.1/runtime/sign.ts
Download https://deno.land/x/jose@v4.13.1/jwt/produce.ts
Download https://deno.land/x/jose@v4.13.1/runtime/digest.ts
Download https://deno.land/x/jose@v4.13.1/runtime/fetch_jwks.ts
Download https://deno.land/x/jose@v4.13.1/runtime/env.ts
Download https://deno.land/x/jose@v4.13.1/runtime/asn1.ts
Download https://deno.land/x/jose@v4.13.1/runtime/key_to_jwk.ts
Download https://deno.land/x/jose@v4.13.1/runtime/jwk_to_key.ts
Download https://deno.land/x/jose@v4.13.1/runtime/generate.ts
Download https://deno.land/x/jose@v4.13.1/lib/check_iv_length.ts
Download https://deno.land/x/jose@v4.13.1/runtime/check_cek_length.ts
Download https://deno.land/x/jose@v4.13.1/runtime/timing_safe_equal.ts
Download https://deno.land/x/jose@v4.13.1/runtime/webcrypto.ts
Download https://deno.land/x/jose@v4.13.1/lib/crypto_key.ts
Download https://deno.land/x/jose@v4.13.1/lib/invalid_key_input.ts
Download https://deno.land/x/jose@v4.13.1/runtime/is_key_like.ts
Download https://deno.land/x/jose@v4.13.1/runtime/aeskw.ts
Download https://deno.land/x/jose@v4.13.1/runtime/ecdhes.ts
Download https://deno.land/x/jose@v4.13.1/runtime/pbes2kw.ts
Download https://deno.land/x/jose@v4.13.1/runtime/rsaes.ts
Download https://deno.land/x/jose@v4.13.1/lib/aesgcmkw.ts
Download https://deno.land/x/jose@v4.13.1/runtime/random.ts
Download https://deno.land/x/jose@v4.13.1/runtime/subtle_dsa.ts
Download https://deno.land/x/jose@v4.13.1/runtime/check_key_length.ts
Download https://deno.land/x/jose@v4.13.1/runtime/get_sign_verify_key.ts
Download https://deno.land/x/jose@v4.13.1/lib/epoch.ts
Download https://deno.land/x/jose@v4.13.1/lib/secs.ts
Download https://deno.land/x/jose@v4.13.1/lib/format_pem.ts
Download https://deno.land/x/jose@v4.13.1/runtime/bogus.ts
Download https://deno.land/x/jose@v4.13.1/lib/check_p2s.ts
Download https://deno.land/x/jose@v4.13.1/runtime/subtle_rsaes.ts
Serving functions on http://127.0.0.1:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.54.6 (compatible with Deno v1.43.0)

@sweatybridge sweatybridge mentioned this issue Jul 6, 2024
Merged
@sweatybridge
Copy link
Contributor

Could you kindly check if this is still reproducible in the beta release?

npx supabase@beta start

@sosensible
Copy link

sosensible commented Jul 12, 2024 •
edited
Loading

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?

config.toml.txt

Got this when health check failed.

Serving functions on http://127.0.0.1:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.55.0 (compatible with Deno v1.43.0)
Stopping containers...
failed to execute http request: Head "http://127.0.0.1:54321/rest-admin/v1/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
failed to execute http request: Head "http://127.0.0.1:54321/functions/v1/_internal/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Try rerunning the command with --debug to troubleshoot the error.

@sweatybridge
Copy link
Contributor

Hmm I'm not sure what's exactly wrong as I can't seem to reproduce locally on macOS. But as a temporary workaround, you can pass in supabase start --ignore-health-check flag.

https://supabase.com/docs/reference/cli/introduction

@Shyrogan
Copy link
Author

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?

config.toml.txt

Got this when health check failed.

Serving functions on http://127.0.0.1:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.55.0 (compatible with Deno v1.43.0)
Stopping containers...
failed to execute http request: Head "http://127.0.0.1:54321/rest-admin/v1/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
failed to execute http request: Head "http://127.0.0.1:54321/functions/v1/_internal/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Try rerunning the command with --debug to troubleshoot the error.

Got it working by recreating the project to default configuration, looks like there is a setting in newer versions blocking us, maybe you could post your project configuration to see ?

@taylorkangbeck
Copy link

Hmm I'm not sure what's exactly wrong as I can't seem to reproduce locally on macOS. But as a temporary workaround, you can pass in supabase start --ignore-health-check flag.

https://supabase.com/docs/reference/cli/introduction

I'm having this same problem, and oddly even with the --ignore-health-check flag, the cli still hangs on Waiting for health checks...

@Shyrogan
Copy link
Author

Shyrogan commented Jul 15, 2024 •
edited
Loading

Hmm I'm not sure what's exactly wrong as I can't seem to reproduce locally on macOS. But as a temporary workaround, you can pass in supabase start --ignore-health-check flag.
https://supabase.com/docs/reference/cli/introduction

I'm having this same problem, and oddly even with the --ignore-health-check flag, the cli still hangs on Waiting for health checks...

Can you try the solution I sent above ?

@rogix
Copy link

rogix commented Jul 17, 2024

Same issue here. I running on macOS 15. Could the issue be related to this macOS version somehow?

@rogix
Copy link

rogix commented Jul 17, 2024

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?
config.toml.txt
Got this when health check failed.

Serving functions on http://127.0.0.1:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.55.0 (compatible with Deno v1.43.0)
Stopping containers...
failed to execute http request: Head "http://127.0.0.1:54321/rest-admin/v1/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
failed to execute http request: Head "http://127.0.0.1:54321/functions/v1/_internal/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Try rerunning the command with --debug to troubleshoot the error.

Got it working by recreating the project to default configuration, looks like there is a setting in newer versions blocking us, maybe you could post your project configuration to see ?

What do you mean by "recreating the project to default configuration"? Would you mind sharing how to do this? Thanks

@sosensible
Copy link

+1 on how you recreated the project to default configuration.

@sweatybridge
Copy link
Contributor

recreated the project to default configuration

You can try these commands to reinitialise the local project config. I'm not 100% sure if it solves the problem though.

supabase stop --no-backup
supabase init --force

@Shyrogan
Copy link
Author

Shyrogan commented Jul 18, 2024 •
edited
Loading

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?
config.toml.txt
Got this when health check failed.

Serving functions on http://127.0.0.1:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.55.0 (compatible with Deno v1.43.0)
Stopping containers...
failed to execute http request: Head "http://127.0.0.1:54321/rest-admin/v1/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
failed to execute http request: Head "http://127.0.0.1:54321/functions/v1/_internal/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Try rerunning the command with --debug to troubleshoot the error.

Got it working by recreating the project to default configuration, looks like there is a setting in newer versions blocking us, maybe you could post your project configuration to see ?

What do you mean by "recreating the project to default configuration"? Would you mind sharing how to do this? Thanks

+1 on how you recreated the project to default configuration.

Simply save your current configuration somewhere and reinitialize it.

Here is the commit that fixed the project on my side (I believe):
Screenshot 2024-07-18 at 09-11-48 chore fix supabase · Shyrogan_mentor@3c57073

@rogix
Copy link

rogix commented Jul 18, 2024

Yep! Just switching enabled to false worked for me. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@sosensible @sweatybridge @taylorkangbeck @rogix @Shyrogan