Fly deploy (launch) of Quickstart app fails

[adamthewebguy]

Create the Quickstart app locally. Everything worked ok.

Decided to attempt deploy to fly (using Fly Postgres app). Launch fails. App status is "suspended".

2024-06-07T18:17:41.567 app[d890ed2a0299e8] lhr [info] WARN Reaped child process with pid: 371 and signal: SIGUSR1, core dumped? false
2024-06-07T18:17:41.683 app[d890ed2a0299e8] lhr [info] 18:17:41.683 pid=<0.3399.0> [info] GET /api/status
2024-06-07T18:17:41.684 app[d890ed2a0299e8] lhr [info] 18:17:41.683 pid=<0.3399.0> [info] Sent 503 in 44µs
2024-06-07T18:17:42.494 app[d890ed2a0299e8] lhr [info] 18:17:42.494 pid=<0.3392.0> [error] Postgrex.Protocol (#PID<0.3392.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (my-app-pg.flycast:5432): non-existing domain - :nxdomain
2024-06-07T18:17:42.757 app[d890ed2a0299e8] lhr [info] 18:17:42.757 pid=<0.3396.0> [error] Postgrex.Protocol (#PID<0.3396.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (my-app-pg.flycast:5432): non-existing domain - :nxdomain
2024-06-07T18:17:42.894 app[d890ed2a0299e8] lhr [info] 18:17:42.894 pid=<0.3389.0> [error] Postgrex.Protocol (#PID<0.3389.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (my-app-pg.flycast:5432): non-existing domain - :nxdomain
2024-06-07T18:17:43.426 app[d890ed2a0299e8] lhr [info] 18:17:43.426 pid=<0.3391.0> [error] Postgrex.Protocol (#PID<0.3391.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (my-app-pg.flycast:5432): non-existing domain - :nxdomain
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] 18:17:43.471 pid=<0.3374.0> origin=postgres_1 [error] PostgresConnectorSup failed to start child Electric.Satellite.ClientReconnectionInfo with reason: {%DBConnection.ConnectionError{message: "connection not available and request was dropped from queue after 2000ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:\n\n 1. Ensuring your database is available and that you can connect to it\n 2. Tracking down slow queries and making sure they are running fast enough\n 3. Increasing the pool_size (although this increases resource consumption)\n 4. Allowing requests to wait longer by increasing :queue_target and :queue_interval\n\nSee DBConnection.start_link/2 for more information\n", severity: :error, reason: :queue_timeout}, [{DBConnection, :run, 3, [file: ~c"lib/db_connection.ex", line: 960, error_info: %{module: Exception}]}, {Electric.Satellite.ClientReconnectionInfo, :init, 1, [file: ~c"lib/electric/satellite/client_reconnection_info.ex", line: 855]}, {:gen_server, :init_it, 2, [file: ~c"gen_server.erl", line: 851]}, {:gen_server, :init_it, 6, [file: ~c"gen_server.erl", line: 814]}, {:proc_lib, :init_p_do_apply, 3, [file: ~c"proc_lib.erl", line: 240]}]}.
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ ┌────────────────────────────────────────────────────────┐
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ │ MODULE ERROR: Electric.Replication.PostgresConnector │
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ ┕━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ Failed to start child Electric.Satellite.ClientReconnectionInfo:
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ {%DBConnection.ConnectionError{message: "connection not available and request was dropped from queue after 2000ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:\n\n 1. Ensuring your database is available and that you can connect to it\n 2. Tracking down slow queries and making sure they are running fast enough\n 3. Increasing the pool_size (although this increases resource consumption)\n 4. Allowing requests to wait longer by increasing :queue_target and :queue_interval\n\nSee DBConnection.start_link/2 for more information\n", severity: :error, reason: :queue_timeout}, [{DBConnection, :run, 3, [file: ~c"lib/db_connection.ex", line: 960, error_info: %{module: Exception}]}, {Electric.Satellite.ClientReconnectionInfo, :init, 1, [file: ~c"lib/electric/satellite/client_reconnection_info.ex", line: 855]}, {:gen_server, :init_it, 2, [file: ~c"gen_server.erl", line: 851]}, {:gen_server, :init_it, 6, [file: ~c"gen_server.erl", line: 814]}, {:proc_lib, :init_p_do_apply, 3, [file: ~c"proc_lib.erl", line: 240]}]}
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ Please file a new issue on GitHub[1], including the contents of this error.
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ▓ [1]: https://github.com/electric-sql/electric/issues
2024-06-07T18:17:43.471 app[d890ed2a0299e8] lhr [info] ••• Shutting down •••

[linear]

VAX-1930 Fly deploy (launch) of Quickstart app fails

[alco]

Hey @adamthewebguy.

Could you share your fly.toml file? I suspect your configuration for Electric is missing the DATABASE_USE_IPV6=true setting.

[adamthewebguy]

Hi @alco , I did ensure the DATABASE_USE_IPV6=true setting was there. Here's the fly.toml file:

# fly.toml app configuration file generated for xxxx-test-app on 2024-06-07T19:33:41+01:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = 'xxxx-test-app'
primary_region = 'lhr'

[build]
  image = 'electricsql/electric:latest'

[env]
  AUTH_MODE = 'insecure'
  DATABASE_REQUIRE_SSL = 'false'
  DATABASE_URL = 'xxxxxxxxx'
  DATABASE_USE_IPV6 = 'true'
  ELECTRIC_WRITE_TO_PG_MODE = 'direct_writes'
  PG_PROXY_PASSWORD = 'proxy_password'

[http_service]
  internal_port = 5133
  force_https = true

  [[http_service.checks]]
    interval = '10s'
    timeout = '1s'
    grace_period = '20s'
    method = 'GET'
    path = '/api/status'

[[services]]
  protocol = 'tcp'
  internal_port = 65432

  [[services.ports]]
    port = 65432
    handlers = ['pg_tls']

[[vm]]
  memory = '1gb'
  cpu_kind = 'shared'
  cpus = 1

[alco]

@adamthewebguy I don't see from the log snippet you shared which version of the sync service it is. We fixed an IPv6-related bug in the most recent release. Can you double-check that you're running that? Perhaps, try deploying with fly deploy --no-cache to force fly to pull the latest Docker image from Docker Hub.

[adamthewebguy]

I set up the Quickstart last Friday (7th June 2024)
So I set up the project with:

npx create-electric-app@latest my-app --template react

Looking in my package.json, I see:

"electric-sql": "^0.10.1",

Is that the sync service that you're referring to?

[alco]

@adamthewebguy what you've shown is the client library. The latest version of that is 0.12.0, released just yesterday.

The sync service is the server component which you deploy to Fly as a Docker image. It is specified in the [build] section of fly.toml:

[build]
  image = 'electricsql/electric:latest'

I hypothesise that Fly may cache the image after it originally fetched it from Docker Hub. To make sure you're using the latest published image, follow the steps from my previous comment or edit your fly.toml file to look like this:

[build]
  image = 'electricsql/electric:0.12'

[adamthewebguy]

Hi @alco
I scrapped the app, launched a fresh one and it appears to deploy ok now.
But when I visit the url I just see "Not found" in the browser window

fly app list shows the app status as "deployed", and
curl https://electricsql-on-fly-test-app.fly.dev/api/status shows: Connection to Postgres is up!%

[thruflo]

That’s correct if you’re opening https://electricsql-on-fly-test-app.fly.dev in the browser.

To see your app you need to deploy the static app and set the config.url in your client https://electric-sql.com/docs/api/clients/typescript#configuration to connect to your sync service running on Fly.

[adamthewebguy]

Thanks @thruflo and @alco

I've gone back and read about the architecture and concepts again. I was following this and thought I'd deployed the Electric part of the backend and the client (React) app with the single fly 'app' deploy. (My excuse - I'm currently working on a full stack project where we do deploy the backend and frontend in one fly deploy and I think that's clouded my thinking here!)

BTW James, I was impressed by your docs, but just in case it helps to clear things up (although, maybe I'm the only one!) then here's what happened in my case:

• Went down the Fly Deployment page to "Deploying Electric connected to Fly Postgres"
• Did that, then clicked on "Now go back to Configure your Fly app"... taking me back up the page
• Completed the section entitled "Deploy!" (that was where I got caught out - this made me think "Cool, I'm done!")
• Only now do I look to the bottom of the page and see Connecting the client app to Electric running on Fly

For me, it would be clearer to remind at the top of the Deployment doc that there are 3 things to be considered:

• Postgres (Fly or external)
• Electric sync service (in this case, a Fly app)
• Client app (local, another app on Fly.. etc.)

Then again, it might just be me that got caught out here 😄

Love what you're doing, all the best!!!

[alco]

@adamthewebguy I'm glad to see you've got your issue resolved. And thanks for the feedback! ❤️