Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot send or receive messages with self-hosted Realtime + hosted Supabase Postgres #372

Closed
Omarzipan opened this issue Dec 9, 2022 · 41 comments
Labels
bug Something isn't working

Comments

@Omarzipan
Copy link

Omarzipan commented Dec 9, 2022

Bug report

Describe the bug

I am running a local web app that connects to a local version of Supabase Realtime running in Docker. The webapp uses supabase/realtime-js to connect to the realtime server. The Realtime server is connected to a hosted Postgres database on Supabase for my project.

This setup works when I point my web app to use Supabase's hosted Realtime servers. However, when I switch my realtime-js client library to point to my local Realtime server instead, I get an error.

On the client, I get a CHANNEL_ERROR and a null error object.

On the Realtime server, I get the following error:

realtime_1  | 2022-12-09 08:12:14.634 [info] CONNECTED TO RealtimeWeb.UserSocket in 64µs
realtime_1  |   Transport: :websocket
realtime_1  |   Serializer: Phoenix.Socket.V1.JSONSerializer
realtime_1  |   Parameters: %{"apikey" => "<Redacted Supabase Anon JWT key>", "eventsPerSecond" => "160", "vsn" => "1.0.0"}
realtime_1  | 2022-12-09 08:12:14.657 [info] JOINED realtime:test-room-2 in 31µs
realtime_1  |   Parameters: %{"config" => %{"broadcast" => %{"ack" => false, "self" => false}, "postgres_changes" => [], "presence" => %{"key" => "k56pkz275fzjq3j7h7o8rlf1lrsbnm"}}}
realtime_1  | 2022-12-09 08:12:14.666 [error] GenServer #PID<0.3943.0> terminating
realtime_1  | ** (FunctionClauseError) no function clause matching in RealtimeWeb.RealtimeChannel.handle_in/3
realtime_1  |     (realtime 0.0.0-automated) lib/realtime_web/channels/realtime_channel.ex:138: RealtimeWeb.RealtimeChannel.handle_in("presence", %{"event" => "track", "payload" => %{"online_at" => "2022-12-09T08:12:14.664Z"}, "type" => "presence"}, %Phoenix.Socket{assigns: %{}, channel: RealtimeWeb.RealtimeChannel, channel_pid: #PID<0.3943.0>, endpoint: RealtimeWeb.Endpoint, handler: RealtimeWeb.UserSocket, id: nil, join_ref: "1", joined: true, private: %{log_handle_in: :debug, log_join: :info}, pubsub_server: Realtime.PubSub, ref: "2", serializer: Phoenix.Socket.V1.JSONSerializer, topic: "realtime:test-room-2", transport: :websocket, transport_pid: #PID<0.3941.0>})
realtime_1  |     (phoenix 1.5.8) lib/phoenix/channel/server.ex:315: Phoenix.Channel.Server.handle_info/2
realtime_1  |     (stdlib 3.14.2.2) gen_server.erl:689: :gen_server.try_dispatch/4
realtime_1  |     (stdlib 3.14.2.2) gen_server.erl:765: :gen_server.handle_msg/6
realtime_1  |     (stdlib 3.14.2.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
realtime_1  | Last message: %Phoenix.Socket.Message{event: "presence", join_ref: "1", payload: %{"event" => "track", "payload" => %{"online_at" => "2022-12-09T08:12:14.664Z"}, "type" => "presence"}, ref: "2", topic: "realtime:test-room-2"}

To Reproduce

On the client I'm using realtime-js:

const localUrl = 'ws://localhost:4000/socket';
const realtimeClient = new RealtimeClient(localUrl, {
  params: {
    apikey: supabaseAnonKey,
    eventsPerSecond: MAX_EVENTS_PER_SECOND,
  },
});

On the backend I'm running the latest version of Realtime's Dockerfile from this repo, but modified the envvars with my Supabase Postgres settings.

version: '3'
services:
  realtime:
    image: supabase/realtime:latest
    build: .
    ports:
      - "4000:4000"
    environment:
      DB_HOST: db.<my project ID from supabase dashboard>.supabase.co
      DB_NAME: postgres
      DB_USER: postgres
      DB_PASSWORD: <my Supabase password>
      DB_PORT: 5432 
      PORT: 4000
      JWT_SECRET: '<Redacted Supabase JWT secret from dashboard>'
      SECURE_CHANNELS: 'false' 
      EXPOSE_METRICS: 'true'
      DB_SSL: 'false' 
      DB_IP_VERSION: 'IPv4'
      REPLICATION_MODE: 'STREAM'
      SLOT_NAME: 'meshi1'
      REALTIME_IP_VERSION: 'IPv6' # was v6
      PUBLICATIONS: "[\"supabase_realtime\"]"
      JWT_CLAIM_VALIDATORS: '{"iss": "Issuer", "nbf": 1610078130}'
      MAX_REPLICATION_LAG_MB: 1000

Expected behavior

I would expect to be able to send and receive messages by using a self-hosted Realtime server with a hosted Supabase database.

Screenshots

n/a

System information

  • OS: macOS + Docker
  • Browser: Chrome
  • Version of supabase-js: "@supabase/realtime-js": "^2.1.0", "@supabase/supabase-js": "^2.0.2",
  • Version of Node.js: 16.15.1
@Omarzipan Omarzipan added the bug Something isn't working label Dec 9, 2022
@chasers
Copy link
Contributor

chasers commented Dec 14, 2022

We have an rc branch here which, when merged, we'll be shipping the latest version of Realtime with the rest of self-hosted Supabase.

Should be very soon!

@Omarzipan
Copy link
Author

that's great! when that's merged will it also be reflected in the latest Docker image?

@thecooltechguy
Copy link

thecooltechguy commented Dec 15, 2022

I'm also facing issues trying to use Realtime supabase locally:

I'm using the latest supabase CLI locally w/ the supabase v2 js libraries in react, and I'm finding that realtime subscriptions to DB events isn't working.

In the logs for the realtime container, I see the following 2 messages repeatedly:

11:39:05.548 [warning] [libcluster:fly6pn] unable to connect to :"realtime@23.217.138.110"
11:39:12.111 project=realtime-demo external_id=realtime-demo [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "INSERT", "schema" => "public", "table" => "chat_messages"}}}

In react, the realtime subscription code looks something like:

        const subscription = supabase
            .channel(`public:chat_messages`)
            .on('postgres_changes', { event: 'INSERT', schema: 'public', table: 'chat_messages' }, payload => {
                console.log("REALTIME!");
                console.log({payload});
            })
            .subscribe()

The subscription object has state set to "joined", and the channel shows up if I subsequently call supabase.getChannels().

But for some reason, the payload function is never called and the websocket messages received by the react client from the websocket server indicate the same error message repeatedly:

{"event":"system","payload":{"channel":"public:chat_messages","extension":"postgres_changes","message":"Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{\"event\" => \"*\", \"schema\" => \"public\", \"table\" => \"chat_messages\"}}}","status":"error"},"ref":null,"topic":"realtime:public:chat_messages"}

Are there any suggested workarounds for local development that needs real-time, until an update is merged? Thanks!

@w3b6x9
Copy link
Member

w3b6x9 commented Dec 16, 2022

@thecooltechguy did you remember to add the chat_messages table to the supabase_realtime publication?

alter publication supabase_realtime add table chat_messages;

@thecooltechguy
Copy link

@w3b6x9 THANK U SO MUCH! It turned out that I forgot to enable "Realtime" for the table after I upgraded my supabase local setup to use the latest container images. Now, realtime works locally!

@chasers chasers closed this as completed Dec 16, 2022
@w3b6x9 w3b6x9 reopened this Dec 16, 2022
@w3b6x9
Copy link
Member

w3b6x9 commented Dec 16, 2022

@Omarzipan try pointing the Realtime image at v1.0.0-rc.14 and using these env vars starting here: https://github.com/supabase/cli/blob/2283024565952c1ae4a6a9bf144f2abe230e4ce4/internal/start/start.go#L305.

I'm currently updating Realtime's docker-compose.yml so this will be much more straightforward.

@bhvngt
Copy link

bhvngt commented Dec 17, 2022

try pointing the Realtime image at v1.0.0-rc.14 and using these env vars starting here: https://github.com/supabase/cli/blob/2283024565952c1ae4a6a9bf144f2abe230e4ce4/internal/start/start.go#L305.

@w3b6x9. I have a docker compose setup. When I upgrade the image to v1.0.0-rc.14 from v1.0.0-rc.12 with the above env vars, I get following warning and error. I am using @supabase/supabase-js v2.2.0

supabase-realtime  | 05:55:13.943 [error] #PID<0.3017.0> running RealtimeWeb.Endpoint (connection #PID<0.3011.0>, stream id 2) terminated
supabase-realtime  | Server: realtime:4000 (http)
supabase-realtime  | Request: GET /socket/websocket?apikey=xxxxxxxxx&vsn=1.0.0
supabase-realtime  | ** (exit) an exception was raised:
supabase-realtime  |     ** (Postgrex.Error) ERROR 42P01 (undefined_table) relation "tenants" does not exist
supabase-realtime  |
supabase-realtime  |     query: SELECT t0."id", t0."name", t0."external_id", t0."jwt_secret", t0."postgres_cdc_default", t0."max_concurrent_users", t0."max_events_per_second", t0."inserted_at", t0."updated_at" FROM "tenants" AS t0 WHERE (t0."external_id" = $1)
supabase-realtime  |         (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:932: Ecto.Adapters.SQL.raise_sql_call_error/1
supabase-realtime  |         (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:847: Ecto.Adapters.SQL.execute/6
supabase-realtime  |         (ecto 3.8.4) lib/ecto/repo/queryable.ex:221: Ecto.Repo.Queryable.execute/4
supabase-realtime  |         (ecto 3.8.4) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
supabase-realtime  |         (ecto 3.8.4) lib/ecto/repo/queryable.ex:147: Ecto.Repo.Queryable.one/3
supabase-realtime  |         (realtime 2.0.0) lib/realtime/api.ex:132: Realtime.Api.get_tenant_by_external_id/1
supabase-realtime  |         (realtime 2.0.0) lib/realtime_web/channels/user_socket.ex:44: RealtimeWeb.UserSocket.connect/3
supabase-realtime  |         (phoenix 1.6.13) lib/phoenix/socket.ex:550: Phoenix.Socket.user_connect/6
supabase-realtime  | 05:55:13.958 [warning] [libcluster:fly6pn] dns polling strategy is selected, but query or basename param is invalid: %{node_basename: "realtime", query: ""}

FYI - tenants is one of my domain table which is not part of realtime publication. I have only one table that is part of realtime publication. My realtime subscription code is not subscibing to tenants

I also keep getting following warning

 06:01:34.137 [warning] [libcluster:fly6pn] dns polling strategy is selected, but query or basename param is invalid: %{node_basename: "realtime", query: ""}

My docker-compose file for realtime is setup as follows

  realtime:
    container_name: supabase-realtime
    image: supabase/realtime:v1.0.0-rc.14
    depends_on:
      - db
    restart: unless-stopped
    environment:
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${PGDATABASE}
      DB_USER: ${PGUSER}
      DB_PASSWORD: ${PGPASSWORD}
      PORT: 4000
      DB_AFTER_CONNECT_QUERY: "SET search_path TO _realtime"
      DB_ENC_KEY: "supabaserealtime"
      FLY_ALLOC_ID: "abc123"
      FLY_APP_NAME: "realtime"
      SECRET_KEY_BASE: "EAx3IQ/wRG1v47ZD4NE4/9RzBI8Jmil3x0yhcW4V2NHBP6c2iPIzwjofi2Ep4HIG"
      ERL_AFLAGS: "-proto_dist inet_tcp"
      ENABLE_TAILSCALE: false
      DNS_NODES: ''

docker compose config with v1.0.0-rc.12 that works well without any code modification is as follows

  realtime:
    container_name: supabase-realtime
    image: supabase/realtime:v1.0.0-rc.12
    depends_on:
      - db
    restart: unless-stopped
    environment:
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: ${PGDATABASE}
      DB_USER: ${PGUSER}
      DB_PASSWORD: ${PGPASSWORD}
      DB_SSL: "false"
      PORT: 4000
      JWT_SECRET: ${JWT_SECRET}
      REPLICATION_MODE: RLS
      REPLICATION_POLL_INTERVAL: 100
      SECURE_CHANNELS: "true"
      SLOT_NAME: supabase_realtime_rls
      TEMPORARY_SLOT: "true"
    command: >
      bash -c "./prod/rel/realtime/bin/realtime eval Realtime.Release.migrate
      && ./prod/rel/realtime/bin/realtime start"

@nuKs
Copy link

nuKs commented Dec 19, 2022

I have the same issue, except 1.0.0-rc.12 (or even 1.0.0-rc.1) doesn't work either: I get supabase-realtime | bash: line 1: ./prod/rel/realtime/bin/realtime: No such file or directory. + issues with ENABLE_TAILSCALE var not set if I remove the docker-compose command expression.

supabase/realtime:v0.25.1 works fine with the old settings but is not compatible with latest other docker images due to missing postgres procedure.

@itisnajim
Copy link

itisnajim commented Dec 19, 2022

@bhvngt @w3b6x9 how did you get supabase/realtime:v1.0.0-rc.12 working?, as @nuKs mention ./prod/rel/realtime/bin/realtime: No such file or directory error will be thrown!

@w3b6x9
Copy link
Member

w3b6x9 commented Dec 19, 2022

Here's what you can do to get it running locally:

  1. Check out branch chore/docker-setup. You can see the changes to docker-compose.yml here: https://github.com/supabase/realtime/pull/377/files#diff-e45e45baeda1c1e73482975a664062aa56f20c03dd9d64a827aba57775bed0d3.
  2. Slightly tweak the docker-compose.yml file so that it points to supabase/realtime:v1.0.0-rc.14:
...

realtime:
    depends_on:
      - db
    image: supabase/realtime:v1.0.0-rc.14
    container_name: realtime-server
    ports:
      - "4000:4000"
    environment:
      PORT: 4000
      DB_HOST: host.docker.internal
      DB_PORT: 5432
      DB_USER: postgres
      DB_PASSWORD: postgres
      DB_NAME: postgres
      DB_ENC_KEY: supabaserealtime
      DB_AFTER_CONNECT_QUERY: 'SET search_path TO _realtime'
      API_JWT_SECRET: dc447559-996d-4761-a306-f47a5eab1623
      FLY_ALLOC_ID: fly123
      FLY_APP_NAME: realtime
      SECRET_KEY_BASE: UpNVntn3cDxHJpq99YMc1T1AQgQpc8kfYTuRgBiYa15BLrx8etQoXz3gZv1/u2oq
      ERL_AFLAGS: -proto_dist inet_tcp
      ENABLE_TAILSCALE: false
      DNS_NODES: "''"
    command: sh -c "/app/bin/migrate && /app/bin/server"
  1. Make a POST request to insert a tenant into the _realtime.tenant table. Supabase Realtime in production has a separate database for all tenants but for local development we created a _realtime to make setup easier. The Authorization token is signed with the secret found in the env var API_JWT_SECRET.
curl -X POST \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiIiLCJpYXQiOjE2NzEyMzc4NzMsImV4cCI6MTcwMjc3Mzk5MywiYXVkIjoiIiwic3ViIjoiIn0._ARixa2KFUVsKBf3UGR90qKLCpGjxhKcXY4akVbmeNQ' \
  -d $'{
    "tenant" : {
      "name": "realtime-dev",
      "external_id": "realtime-dev",
      "jwt_secret": "a1d99c8b-91b6-47b2-8f3c-aa7d9a9ad20f",
      "extensions": [
        {
          "type": "postgres_cdc_rls",
          "settings": {
            "db_name": "postgres",
            "db_host": "host.docker.internal",
            "db_user": "postgres",
            "db_password": "postgres",
            "db_port": "5432",
            "region": "us-west-1",
            "poll_interval_ms": 100,
            "poll_max_record_bytes": 1048576,
            "ip_version": 4
          }
        }
      ]
    }
  }' \
  http://localhost:4000/api/tenants
  1. Create a table in the database that you want Realtime to listen to.
  2. Use a client like @supabase/realtime-js or go to http://localhost:4000/inspector/new. If using the Realtime Inspector, set the Path to ws://realtime-dev.localhost:4000/socket and Token to eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDMwMjgwODcsInJvbGUiOiJwb3N0Z3JlcyJ9.tz_XJ89gd6bN8MBpCl7afvPrZiBH6RB65iA1FadPT3Y (signed with jwt_secret as part of POST request).

I'm in the process of updating the README and will include the above instructions for easier local setup.

@bhvngt
Copy link

bhvngt commented Dec 20, 2022

THanks @w3b6x9

After making these changes I am getting following error

supabase-realtime  | 01:56:20.978 project=realtime external_id=realtime [error] Auth error: tenant `realtime` not found

Here's my docker-compose file. I had to change DB_HOST to db since my postgres is running as db service

  db:
    container_name: supabase-db
    image: supabase/postgres:15.1.0.14
    command: >
      postgres -c config_file=/etc/postgresql/postgresql.conf 
      -c log_statement=all -c logging_collector=on 
      -c log_destination=csvlog -c log_directory=/var/log/postgresql
      -c log_filename=postgresql -c log_min_messages=fatal
    restart: unless-stopped
    ports:
      - ${PGPORT}:5432
    environment:
      POSTGRES_PASSWORD: ${PGPASSWORD}
    volumes:
      - /var/lib/postgresql/data
      - ./volumes/db/init:/docker-entrypoint-initdb.d
      - ${PROJECT_ROOT}/data/supabase/schema:/schema

  realtime:
    container_name: supabase-realtime
    image: supabase/realtime:v1.0.0-rc.14
    depends_on:
      - db
    restart: unless-stopped
    environment:
      PORT: 4000
      DB_HOST: db
      DB_PORT: 5432
      DB_USER: postgres
      DB_PASSWORD: postgres
      DB_NAME: postgres
      DB_ENC_KEY: supabaserealtime
      DB_AFTER_CONNECT_QUERY: 'SET search_path TO _realtime'
      API_JWT_SECRET: ${JWT_SECRET}
      FLY_ALLOC_ID: fly123
      FLY_APP_NAME: realtime
      SECRET_KEY_BASE: UpNVntn3cDxHJpq99YMc1T1AQgQpc8kfYTuRgBiYa15BLrx8etQoXz3gZv1/u2oq
      ERL_AFLAGS: -proto_dist inet_tcp
      ENABLE_TAILSCALE: false
      DNS_NODES: "''"
    command: sh -c "/app/bin/migrate && /app/bin/server"

I have ran following sql to my db. When I ran curl command to post data through postgrest, I got following error - "message": "no Route matched with those values". This could be due to _realtime not being part of PGRST_DB_SCHEMAS

INSERT INTO _realtime.tenants
    (id, name, external_id, jwt_secret, inserted_at, updated_at)
VALUES (extensions.uuid_generate_v4(), 'realtime-dev', 'realtime-dev',
        'your-super-secret-jwt-token-with-at-least-32-characters-long', NOW(), NOW());

INSERT INTO _realtime.extensions
    (id, type, settings, tenant_external_id, inserted_at, updated_at)
VALUES (extensions.uuid_generate_v4(), 'postgres_cdc_rls',
        json_build_object(
                'db_name', 'postgres',
                'db_host', 'db',
                'db_user', 'postgres',
                'db_password', 'postgres',
                'db_port', '5432',
                'poll_interval_ms', 100,
                'poll_max_record_bytes', 1048576,
                'ip_version', 4
            ), 'realtime-dev', NOW(), NOW())

my supabase-js client code looks like this

    supabaseClient.channel(`${schema}:test`)
                             .on("postgres_changes", { event: "*", schema: schema, table: 'test' },
                               (payload) => console.log(payload)
                             )
                             .subscribe();

@w3b6x9
Copy link
Member

w3b6x9 commented Dec 20, 2022

After making these changes I am getting following error

supabase-realtime  | 01:56:20.978 project=realtime external_id=realtime [error] Auth error: tenant `realtime` not found

@bhvngt hmm it's strange that the external_id is realtime when I've outlined realtime-dev above. Are you using some older version of the CLI and running Realtime that way? There was an issue I fixed in the CLI by renaming the external_id from realtime.

I have ran following sql to my db. When I ran curl command to post data through postgrest, I got following error - "message": "no Route matched with those values". This could be due to _realtime not being part of PGRST_DB_SCHEMAS

@bhvngt ah, the curl command was for http://localhost:4000/api/tenants endpoint and it won't work when trying to hit PostgREST.

@w3b6x9
Copy link
Member

w3b6x9 commented Dec 20, 2022

Here's what you can do to get it running locally:

  1. Check out branch chore/docker-setup. You can see the changes to docker-compose.yml here: https://github.com/supabase/realtime/pull/377/files#diff-e45e45baeda1c1e73482975a664062aa56f20c03dd9d64a827aba57775bed0d3.
  2. Slightly tweak the docker-compose.yml file so that it points to supabase/realtime:v1.0.0-rc.14:
...

realtime:
    depends_on:
      - db
    image: supabase/realtime:v1.0.0-rc.14
    container_name: realtime-server
    ports:
      - "4000:4000"
    environment:
      PORT: 4000
      DB_HOST: host.docker.internal
      DB_PORT: 5432
      DB_USER: postgres
      DB_PASSWORD: postgres
      DB_NAME: postgres
      DB_ENC_KEY: supabaserealtime
      DB_AFTER_CONNECT_QUERY: 'SET search_path TO _realtime'
      API_JWT_SECRET: dc447559-996d-4761-a306-f47a5eab1623
      FLY_ALLOC_ID: fly123
      FLY_APP_NAME: realtime
      SECRET_KEY_BASE: UpNVntn3cDxHJpq99YMc1T1AQgQpc8kfYTuRgBiYa15BLrx8etQoXz3gZv1/u2oq
      ERL_AFLAGS: -proto_dist inet_tcp
      ENABLE_TAILSCALE: false
      DNS_NODES: "''"
    command: sh -c "/app/bin/migrate && /app/bin/server"
  1. Make a POST request to insert a tenant into the _realtime.tenant table. Supabase Realtime in production has a separate database for all tenants but for local development we created a _realtime to make setup easier. The Authorization token is signed with the secret found in the env var API_JWT_SECRET.
curl -X POST \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiIiLCJpYXQiOjE2NzEyMzc4NzMsImV4cCI6MTcwMjc3Mzk5MywiYXVkIjoiIiwic3ViIjoiIn0._ARixa2KFUVsKBf3UGR90qKLCpGjxhKcXY4akVbmeNQ' \
  -d $'{
    "tenant" : {
      "name": "realtime-dev",
      "external_id": "realtime-dev",
      "jwt_secret": "a1d99c8b-91b6-47b2-8f3c-aa7d9a9ad20f",
      "extensions": [
        {
          "type": "postgres_cdc_rls",
          "settings": {
            "db_name": "postgres",
            "db_host": "host.docker.internal",
            "db_user": "postgres",
            "db_password": "postgres",
            "db_port": "5432",
            "region": "us-west-1",
            "poll_interval_ms": 100,
            "poll_max_record_bytes": 1048576,
            "ip_version": 4
          }
        }
      ]
    }
  }' \
  http://localhost:4000/api/tenants
  1. Create a table in the database that you want Realtime to listen to.
  2. Use a client like @supabase/realtime-js or go to http://localhost:4000/inspector/new. If using the Realtime Inspector, set the Path to ws://realtime-dev.localhost:4000/socket and Token to eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDMwMjgwODcsInJvbGUiOiJwb3N0Z3JlcyJ9.tz_XJ89gd6bN8MBpCl7afvPrZiBH6RB65iA1FadPT3Y (signed with jwt_secret as part of POST request).

I'm in the process of updating the README and will include the above instructions for easier local setup.

I've streamlined this process further so now you don't have to run step 3 (POST request). Instead, just pull down chore/docker-setup branch where 15de9a9 will insert into _realtime.tenants for you.

@bhvngt
Copy link

bhvngt commented Dec 20, 2022

Thanks @w3b6x9.

Thanks @w3b6x9

@bhvngt ah, the curl command was for http://localhost:4000/api/tenants endpoint and it won't work when trying to hit PostgREST.

I got that. Now I have it working.

@bhvngt hmm it's strange that the external_id is realtime when I've outlined realtime-dev above. Are you using some older version of the CLI and running Realtime that way? There was an issue I fixed in the CLI by renaming the external_id from realtime.

I am not using supabase-cli and instead using docker compose directly with following config.

For Step 5. Can I use @supabase/supabase-js or do I need to use @supabase/realtime-js or phoenix.
In case if @supabase/supabase-js v2 is fine then do I need to make any changes inside kong.yaml to configure kong proxy?

  kong:
    container_name: supabase-kong
    image: kong:2.1
    restart: unless-stopped
    ports:
      - ${KONG_HTTP_PORT}:8000/tcp
      - ${KONG_HTTPS_PORT}:8443/tcp
    environment:
      KONG_DATABASE: "off"
      KONG_DECLARATIVE_CONFIG: /var/lib/kong/kong.yml
      # https://github.com/supabase/cli/issues/14
      KONG_DNS_ORDER: LAST,A,CNAME
      KONG_PLUGINS: request-transformer,cors,key-auth,acl
    volumes:
      - ./volumes/api/kong.yml:/var/lib/kong/kong.yml
  realtime:
    container_name: supabase-realtime
    image: supabase/realtime:v1.0.0-rc.14
    depends_on:
      - db
    restart: unless-stopped
    ports:
      - "4100:4000"
    environment:
      PORT: 4000
      DB_HOST: db
      DB_PORT: 5432
      DB_USER: postgres
      DB_PASSWORD: postgres
      DB_NAME: postgres
      DB_ENC_KEY: supabaserealtime
      DB_AFTER_CONNECT_QUERY: 'SET search_path TO _realtime'
      API_JWT_SECRET: your-super-secret-jwt-token-with-at-least-32-characters-long
      FLY_ALLOC_ID: fly123
      FLY_APP_NAME: realtime
      SECRET_KEY_BASE: UpNVntn3cDxHJpq99YMc1T1AQgQpc8kfYTuRgBiYa15BLrx8etQoXz3gZv1/u2oq
      ERL_AFLAGS: -proto_dist inet_tcp
      ENABLE_TAILSCALE: false
      DNS_NODES: "''"
    command: sh -c "/app/bin/migrate && /app/bin/server"
  db:
    container_name: supabase-db
    image: supabase/postgres:15.1.0.14
    command: >
      postgres -c config_file=/etc/postgresql/postgresql.conf 
      -c log_statement=all -c logging_collector=on 
      -c log_destination=csvlog -c log_directory=/var/log/postgresql
      -c log_filename=postgresql -c log_min_messages=fatal
    restart: unless-stopped
    ports:
      - ${PGPORT}:5432
    environment:
      POSTGRES_PASSWORD: ${PGPASSWORD}
    volumes:
      - /var/lib/postgresql/data
      - ./volumes/db/init:/docker-entrypoint-initdb.d
      - ${PROJECT_ROOT}/data/supabase/schema:/schema

Here's the kong.yaml realtime config

  - name: realtime-v1
    _comment: "Realtime: /realtime/v1/* -> ws://realtime:4000/socket/*"
    url: http://realtime:4100/socket/
    routes:
      - name: realtime-v1-all
        strip_path: true
        paths:
          - /realtime/v1/
    plugins:
      - name: cors
      - name: key-auth
        config:
          hide_credentials: false
      - name: acl
        config:
          hide_groups_header: true
          allow:
            - admin
            - anon

@w3b6x9
Copy link
Member

w3b6x9 commented Dec 20, 2022

Can I use @supabase/supabase-js or do I need to use @supabase/realtime-js

You can use either.

do I need to make any changes inside kong.yaml to configure kong proxy?

Here's the kong.yaml realtime config

  - name: realtime-v1
    _comment: "Realtime: /realtime/v1/* -> ws://realtime:4000/socket/*"
    url: http://realtime:4100/socket/
    routes:
      - name: realtime-v1-all
        strip_path: true
        paths:
          - /realtime/v1/
    plugins:
      - name: cors
      - name: key-auth
        config:
          hide_credentials: false
      - name: acl
        config:
          hide_groups_header: true
          allow:
            - admin
            - anon

Yes, you'll have to update Realtime url to something like http://[tenant].realtime:4100/socket. So in the case of the instructions I posted above it'll have to be http://realtime-dev.realtime:4100/socket assuming you changed the port to 4100 and renamed Realtime Docker server host to realtime.

@itisnajim
Copy link

@w3b6x9 after following the yml configs i have this now:

supabase-realtime  | 10:37:22.565 [error] Could not create schema migrations table. This error usually happens due to the following:
supabase-realtime  | 
supabase-realtime  |   * The database does not exist
supabase-realtime  |   * The "schema_migrations" table, which Ecto uses for managing
supabase-realtime  |     migrations, was defined by another library
supabase-realtime  |   * There is a deadlock while migrating (such as using concurrent
supabase-realtime  |     indexes with a migration_lock)
supabase-realtime  | 
supabase-realtime  | To fix the first issue, run "mix ecto.create".

am i missing something?

@w3b6x9
Copy link
Member

w3b6x9 commented Dec 20, 2022

@itisnajim can you share your steps so I can try and reproduce?

@itisnajim
Copy link

itisnajim commented Dec 20, 2022

@w3b6x9 here is my full yml configs:

docker-compose.yml

version: "3.8"

services:
  studio:
    container_name: supabase-studio
    image: supabase/studio:20221214-4eecc99
    restart: unless-stopped
    ports:
      - ${STUDIO_PORT}:3000/tcp
    environment:
      STUDIO_PG_META_URL: http://meta:8080
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

      DEFAULT_ORGANIZATION: ${STUDIO_DEFAULT_ORGANIZATION}
      DEFAULT_PROJECT: ${STUDIO_DEFAULT_PROJECT}

      SUPABASE_URL: http://kong:8000
      SUPABASE_PUBLIC_URL: ${SUPABASE_PUBLIC_URL}
      # Kept for backwards compatibility with studio:0.22.08
            SUPABASE_REST_URL: ${API_EXTERNAL_URL}/rest/v1/
      SUPABASE_ANON_KEY: ${ANON_KEY}
      SUPABASE_SERVICE_KEY: ${SERVICE_ROLE_KEY}

  kong:
    container_name: supabase-kong
    image: kong:2.8.1
    restart: unless-stopped
    ports:
      - ${KONG_HTTP_PORT}:8000/tcp
      - ${KONG_HTTPS_PORT}:8443/tcp
    environment:
      KONG_DATABASE: "off"
      KONG_DECLARATIVE_CONFIG: /var/lib/kong/kong.yml
      # https://github.com/supabase/cli/issues/14
      KONG_DNS_ORDER: LAST,A,CNAME
      KONG_PLUGINS: request-transformer,cors,key-auth,acl
      KONG_NGINX_PROXY_PROXY_BUFFER_SIZE: 160k
      KONG_NGINX_PROXY_PROXY_BUFFERS: 64 160k
    volumes:
      - ./volumes/api:/var/lib/kong:ro

  auth:
    container_name: supabase-auth
    image: supabase/gotrue:v2.31.0
    depends_on:
      db: # Disable this if you are using an external Postgres database
        condition: service_healthy
    restart: unless-stopped
    environment:
      GOTRUE_API_HOST: 0.0.0.0
      GOTRUE_API_PORT: 9999
      API_EXTERNAL_URL: ${API_EXTERNAL_URL}

      GOTRUE_DB_DRIVER: postgres
      GOTRUE_DB_DATABASE_URL: postgres://supabase_auth_admin:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POST>

      GOTRUE_SITE_URL: ${SITE_URL}
      GOTRUE_URI_ALLOW_LIST: ${ADDITIONAL_REDIRECT_URLS}
      GOTRUE_DISABLE_SIGNUP: ${DISABLE_SIGNUP}

      GOTRUE_JWT_ADMIN_ROLES: service_role
      GOTRUE_JWT_AUD: authenticated
      GOTRUE_JWT_DEFAULT_GROUP_NAME: authenticated
      GOTRUE_JWT_EXP: ${JWT_EXPIRY}
      GOTRUE_JWT_SECRET: ${JWT_SECRET}

      GOTRUE_EXTERNAL_EMAIL_ENABLED: ${ENABLE_EMAIL_SIGNUP}
      GOTRUE_MAILER_AUTOCONFIRM: ${ENABLE_EMAIL_AUTOCONFIRM}
      # GOTRUE_MAILER_SECURE_EMAIL_CHANGE_ENABLED: true
      # GOTRUE_SMTP_MAX_FREQUENCY: 1s
      GOTRUE_SMTP_ADMIN_EMAIL: ${SMTP_ADMIN_EMAIL}
      GOTRUE_SMTP_HOST: ${SMTP_HOST}
      GOTRUE_SMTP_PORT: ${SMTP_PORT}
      GOTRUE_SMTP_USER: ${SMTP_USER}
      GOTRUE_SMTP_PASS: ${SMTP_PASS}
      GOTRUE_SMTP_SENDER_NAME: ${SMTP_SENDER_NAME}
      GOTRUE_MAILER_URLPATHS_INVITE: ${MAILER_URLPATHS_INVITE}
      GOTRUE_MAILER_URLPATHS_CONFIRMATION: ${MAILER_URLPATHS_CONFIRMATION}
      GOTRUE_MAILER_URLPATHS_RECOVERY: ${MAILER_URLPATHS_RECOVERY}
      GOTRUE_MAILER_URLPATHS_EMAIL_CHANGE: ${MAILER_URLPATHS_EMAIL_CHANGE}

      GOTRUE_EXTERNAL_PHONE_ENABLED: ${ENABLE_PHONE_SIGNUP}
      GOTRUE_SMS_AUTOCONFIRM: ${ENABLE_PHONE_AUTOCONFIRM}
      MFA_ENABLED: ${MFA_ENABLED}

  rest:
    container_name: supabase-rest
    image: postgrest/postgrest:v9.0.1.20220717
    depends_on:
      db: # Disable this if you are using an external Postgres database
        condition: service_healthy
    restart: unless-stopped
    environment:
      PGRST_DB_URI: postgres://authenticator:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POS>
      PGRST_DB_SCHEMAS: ${PGRST_DB_SCHEMAS}
      PGRST_DB_ANON_ROLE: anon
      PGRST_JWT_SECRET: ${JWT_SECRET}
      PGRST_DB_USE_LEGACY_GUCS: "false"

  realtime:
    container_name: supabase-realtime
    image: supabase/realtime:v1.0.0-rc.14
    depends_on:
      db: # Disable this if you are using an external Postgres database
        condition: service_healthy
    restart: unless-stopped
    ports:
      - "4000:4000"
    environment:
      DB_HOST: ${POSTGRES_HOST}
      DB_PORT: ${POSTGRES_PORT}
      DB_NAME: ${POSTGRES_DB}
      DB_USER: supabase_admin
      DB_PASSWORD: ${POSTGRES_PASSWORD}
      DB_SSL: "false"
      PORT: 4000
      JWT_SECRET: ${JWT_SECRET}
      REPLICATION_MODE: RLS
      REPLICATION_POLL_INTERVAL: 100
      SECURE_CHANNELS: "true"
      SLOT_NAME: supabase_realtime_rls
      TEMPORARY_SLOT: "true"
      DB_AFTER_CONNECT_QUERY: "SET search_path TO _realtime"
      DB_ENC_KEY: supabaserealtime
      FLY_ALLOC_ID: fly123
      FLY_APP_NAME: realtime
      SECRET_KEY_BASE: UpNVntn3cDxHJpq99YMc1T1AQgQpc8kfYTuRgBiYa15BLrx8etQoXz3gZv1/u2oq
      ERL_AFLAGS: -proto_dist inet_tcp
      ENABLE_TAILSCALE: false
      DNS_NODES: "''"
    command: sh -c "/app/bin/migrate && /app/bin/server"
#    command: >
#      bash -c "./prod/rel/realtime/bin/realtime eval Realtime.Release.migrate
#      && ./prod/rel/realtime/bin/realtime start"

  storage:
    container_name: supabase-storage
    image: supabase/storage-api:v0.26.1
    depends_on:
      db: # Disable this if you are using an external Postgres database
        condition: service_healthy
      rest:
        condition: service_started
    restart: unless-stopped
    environment:
      ANON_KEY: ${ANON_KEY}
      SERVICE_KEY: ${SERVICE_ROLE_KEY}
      POSTGREST_URL: http://rest:3000
      PGRST_JWT_SECRET: ${JWT_SECRET}
      DATABASE_URL: postgres://supabase_storage_admin:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PO>
      FILE_SIZE_LIMIT: 52428800
      STORAGE_BACKEND: file
      FILE_STORAGE_BACKEND_PATH: /var/lib/storage
      TENANT_ID: stub
      # TODO: https://github.com/supabase/storage-api/issues/55
      REGION: stub
      GLOBAL_S3_BUCKET: stub
      ENABLE_IMAGE_TRANSFORMATION: "true"
      IMGPROXY_URL: http://imgproxy:5001
    volumes:
      - ./volumes/storage:/var/lib/storage

  imgproxy:
    container_name: supabase-imgproxy
    image: darthsim/imgproxy:v3.11
    environment:
      IMGPROXY_BIND: ":5001"
      IMGPROXY_LOCAL_FILESYSTEM_ROOT: /
      IMGPROXY_USE_ETAG: "true"
    volumes:
      - ./volumes/storage:/var/lib/storage

  meta:
    container_name: supabase-meta
    image: supabase/postgres-meta:v0.52.1
    depends_on:
      db: # Disable this if you are using an external Postgres database
        condition: service_healthy
    restart: unless-stopped
    environment:
      PG_META_PORT: 8080
      PG_META_DB_HOST: ${POSTGRES_HOST}
      PG_META_DB_PORT: ${POSTGRES_PORT}
      PG_META_DB_NAME: ${POSTGRES_DB}
      PG_META_DB_USER: supabase_admin
      PG_META_DB_PASSWORD: ${POSTGRES_PASSWORD}

  # Comment out everything below this point if you are using an external Postgres database
  db:
    container_name: supabase-db
    image: supabase/postgres:14.1.0.89
    healthcheck:
      test: pg_isready -U postgres -h localhost
      interval: 5s
      timeout: 5s
      retries: 10
    command:
      - postgres
      - -c
      - config_file=/etc/postgresql/postgresql.conf
      - -c
      - log_min_messages=fatal # prevents Realtime polling queries from appearing in logs
    restart: unless-stopped
    ports:
      - ${POSTGRES_PORT}:5432
    environment:
      POSTGRES_HOST: /var/run/postgresql
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - ./volumes/db/roles.sql:/docker-entrypoint-initdb.d/roles.sql

volumes/api/kong.yml

  ## Secure Realtime routes
  - name: realtime-v1
    _comment: "Realtime: /realtime/v1/* -> ws://realtime:4000/socket/*"
    url: http://realtime-dev.realtime:4000/socket
#    url: http://realtime:4000/socket/
    routes:
      - name: realtime-v1-all
        strip_path: true
        paths:
          - /realtime/v1/
    plugins:
      - name: cors
      - name: key-auth
        config:
          hide_credentials: false
      - name: acl
        config:
          hide_groups_header: true
          allow:
            - admin
            - anon

@bhvngt
Copy link

bhvngt commented Dec 20, 2022

Yes, you'll have to update Realtime url to something like http://[tenant].realtime:4100/socket. So in the case of the instructions I posted above it'll have to be http://realtime-dev.realtime:4100/socket assuming you changed the port to 4100 and renamed Realtime Docker server host to realtime.

@w3b6x9 I could make it work by creating RealttimeClient with http://realtime-dev.localhost:4100/socket. For this I had to add realtime-dev.locahost alias in my /etc/hosts. However, when I use @supabase/supabase-js I am stuck with kong not able to forward request to realtime container. It throws following error.

supabase-kong  | 2022/12/20 10:03:38 [error] 22#0: *882 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.96.1, server: kong, request: "GET /realtime/v1/websocket?apikey=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY&vsn=1.0.0 HTTP/1.1", upstream: "http://192.168.96.9:4100/socket/websocket?apikey=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY&vsn=1.0.0", host: "localhost:4000"

Any thoughts on what I need to change in the kong config to make this work?

My kong configuration is

  - name: realtime-v1
    _comment: "Realtime: /realtime/v1/* -> ws://realtime:4100/socket/*"
    url: http://realtime-dev.realtime:4100/socket/
    routes:
      - name: realtime-v1-all
        strip_path: true
        paths:
          - /realtime/v1/
    plugins:
      - name: cors
      - name: key-auth
        config:
          hide_credentials: false
      - name: acl
        config:
          hide_groups_header: true
          allow:
            - admin
            - anon

and my test code is as follows

import {createClient} from '@supabase/supabase-js'

const client = createClient(
    "http://localhost:4000", "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY"
)

// This code works
// import { RealtimeClient } from '@supabase/supabase-js'
//
// const client = new RealtimeClient("http://realtime-dev.localhost:4100/socket", {
//     params: {
//         apikey: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY",
//         eventsPerSecond: 10,
//     },
// })

client.channel('*')
    .on('postgres_changes', {event: '*', schema: 'public'}, (payload) => {
        console.log('All changes in public schema: ', payload)
    })
    .subscribe(async (status) => {
        if (status === 'SUBSCRIBED') {
            console.log(status)
        }
    });

@itisnajim
Copy link

@w3b6x9 I could make it work by creating RealttimeClient with http://realtime-dev.localhost:4100/socket. For this I had to add realtime-dev.locahost alias in my /etc/hosts. However, when I use @supabase/supabase-js I am stuck with kong not able to forward request to realtime container. It throws following error.

supabase-kong  | 2022/12/20 10:03:38 [error] 22#0: *882 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.96.1, server: kong, request: "GET /realtime/v1/websocket?apikey=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY&vsn=1.0.0 HTTP/1.1", upstream: "http://192.168.96.9:4100/socket/websocket?apikey=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY&vsn=1.0.0", host: "localhost:4000"

getting the same error:

supabase-kong      | 2022/12/20 12:24:46 [error] 1110#0: *7798 [lua] init.lua:310: execute(): DNS resolution failed: dns server
 error: 2 server failure. Tried: ["(short)realtime-dev.realtime:(na) - cache-miss","realtime-dev.realtime:1 - cache-
 miss/scheduled/querying/dns server error: 2 server failure","realtime-dev.realtime:5 - cache-miss/scheduled/querying/dns 
 server error: 2 server failure"], client: 172.18.0.1, server: kong, request: "GET /realtime/v1/websocket?
 apikey=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.ewogICAgInJvbGUiOiAiYW5vbiIsCiAgICAiaXNzIjogInN1cGFiYXNlIiwKICAgIC
 JpYXQiOiAxNjcxNDA0NDAwLAogICAgImV4cCI6IDE4MjkxNzA4MDAKfQ.udoS9BfcrvA-
 XIW5FrPLUYtnJIndKuDtX7zpCdXlrnE&vsn=1.0.0 HTTP/1.1", host: "supabase.example.com"

@itisnajim
Copy link

@bhvngt how did you manage to fix this

supabase-realtime  | bash: line 1: ./prod/rel/realtime/bin/realtime: No such file or directory

using supabase/realtime:v1.0.0-rc.12 ?
the error occur because of the command

    command: >
      bash -c "./prod/rel/realtime/bin/realtime eval Realtime.Release.migrate
      && ./prod/rel/realtime/bin/realtime start"

@bhvngt
Copy link

bhvngt commented Dec 20, 2022

@itisnajim. Thats a good question, I did a check with my docker image digest and have since discovered that my image was incorrectly tagged. I was using v0.25.1. So, on pull of v1.0.0-rc.12 I get the same issue.

@itisnajim
Copy link

@w3b6x9 wondering if you were able to download the self-host Supabase docker and start it with the modifications proposed in this issue, I am currently trying to do the same and have been encountering some difficulties. I was wondering if you could share any tips or guidance on how you were able to get it working. Thanks 🙏

@bhvngt
Copy link

bhvngt commented Dec 21, 2022

@w3b6x9 @itisnajim

Its finally working for me. Here's my docker-compose and here's my kong.yaml config to make it work.

Here are the steps I took

  1. Run docker compose
  2. Create tenant by making an API call - ref
  3. Ran following test file
import {createClient} from '@supabase/supabase-js'

const client = createClient(
    "http://localhost:4000",
    "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY"
)

// This will also work but will require change in /etc/hosts to include realtime-dev.localhost as an alias to localhost

// import { RealtimeClient } from '@supabase/realtime-js'
//
// const client = new RealtimeClient("http://realtime-dev.localhost:4100/socket", {
//     params: {
//         apikey: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYyNzIwODU0MCwiZXhwIjoxOTc0MzYzNzQwfQ.zcaQfHd3VA7XgJmdGfmV86OLVJT9s2MTmSy-e69BpUY",
//         eventsPerSecond: 10,
//     },
// })

client.channel('*')
    .on('postgres_changes', {event: '*', schema: 'public'}, (payload) => {
        console.log('All changes in public schema: ', payload)
    })
    .subscribe(async (status) => {
        if (status === 'SUBSCRIBED') {
            console.log(status)
        }
    });

Hopefully after the release of the next version of realtime image, step 2 may not be required.

Key thing to note is that we need to give a separate hostname to realtime container that contains tenant name as a subdomain. For ex - hostname: realtime-dev.realtime and change the url of the kong realtime service to http://realtime-dev.realtime:4000/socket/.

@itisnajim
Copy link

i still have one error, i think it's because /app/bin/migrate on realtime
command: sh -c "/app/bin/migrate && /app/bin/server"

supabase-realtime  | ** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) no schema has been selected to create in
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:932: Ecto.Adapters.SQL.raise_sql_call_error/1
supabase-realtime  |     (elixir 1.14.0) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:1024: Ecto.Adapters.SQL.execute_ddl/4
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:696: Ecto.Migrator.verbose_schema_migration/3
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:510: Ecto.Migrator.lock_for_migrations/4
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:422: Ecto.Migrator.run/4
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:146: Ecto.Migrator.with_repo/3
supabase-realtime  |     nofile:1: (file)
supabase-realtime exited with code 1

@bhvngt you have 3 volumes on db
volumes:
- /var/lib/postgresql/data
- ./volumes/db/init:/docker-entrypoint-initdb.d
- ${PROJECT_ROOT}/data/supabase/schema:/schema

i don't know the content of this ${PROJECT_ROOT}/data/supabase/schema, if you can help me

@bhvngt
Copy link

bhvngt commented Dec 22, 2022

i don't know the content of this ${PROJECT_ROOT}/data/supabase/schema, if you can help me

@itisnajim, it is project specific schema volume. So wont be required for your realtime setup.

@itisnajim
Copy link

mmm 🤔
its a fresh project downloaded directly from the repo supabase/supabase,
@w3b6x9 what do you think about this: ?

i still have one error, i think it's because /app/bin/migrate on realtime command: sh -c "/app/bin/migrate && /app/bin/server"

supabase-realtime  | ** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) no schema has been selected to create in
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:932: Ecto.Adapters.SQL.raise_sql_call_error/1
supabase-realtime  |     (elixir 1.14.0) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:1024: Ecto.Adapters.SQL.execute_ddl/4
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:696: Ecto.Migrator.verbose_schema_migration/3
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:510: Ecto.Migrator.lock_for_migrations/4
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:422: Ecto.Migrator.run/4
supabase-realtime  |     (ecto_sql 3.8.3) lib/ecto/migrator.ex:146: Ecto.Migrator.with_repo/3
supabase-realtime  |     nofile:1: (file)
supabase-realtime exited with code 1

@bhvngt you have 3 volumes on db volumes: - /var/lib/postgresql/data - ./volumes/db/init:/docker-entrypoint-initdb.d - ${PROJECT_ROOT}/data/supabase/schema:/schema

i don't know the content of this ${PROJECT_ROOT}/data/supabase/schema, if you can help me

@Omarzipan
Copy link
Author

Ok, I had time to get back to this. AFAICT the current docker quickstart still has one bug for running things locally. If you run everything normally, you'll get an error about a missing schema:

supabase-realtime  | ** (Postgrex.Error) ERROR 3F000 (invalid_schema_name) no schema has been selected to create in
supabase-realtime exited with code 1

After some sleuthing it's because the _realtime schema was not created, and when the initial migration command is run it fails. This schema should have been created when 00-setup.sql is run, but it's not (& I'm not sure where that should be happening, I couldn't find any place where it was being called).

To fix this:

  1. I modified the docker-compose.yml to only bring up the db service by removing the depends_on
  2. I brought up the postgres container individually (docker-compose -f docker-compose.yml up db)
  3. I connected to the postgres DB with psql and and executed all of the queries in 00-setup.sql
  4. I brought up the realtime server individually ((docker-compose -f docker-compose.yml up realtime)

Everything now works locally!

cc @itisnajim since one of your posts had this error

Originally, I wanted to self-host Realtime and hook it up to a hosted Supabase DB. I'm still a little confused as to how to do that. It seems like I'll need to create a tenant on the hosted Supabase DB, which I should be able to do via an API call to my self-hosted Realtime server? And is there a particular value for DB_ENC_KEY I should be using?

Since I mainly wanted to self-host this to get around limits with just the streaming functionality of Realtime (not trying to subscribe to table updates) I don't mind setting up a dummy DB on some cloud provider and using the current Docker setup. Since it seems like most people here want to self-host both the DB and Realtime, I'm happy to consider this closed and re-open a new issue for just the Self-Hosted Realtime <> Hosted Postgres steps!

@bhvngt
Copy link

bhvngt commented Dec 23, 2022

Thanks @Omarzipan for highlighting the cause of the error. I missed mentioning that I had to change the files kept under volumes/db/init so that I can create the _realtime schema. This gets picked up while initialising postgres container

Here's the script I have added

create schema if not exists _realtime;
create schema if not exists realtime;

create publication supabase_realtime with (publish = 'insert, update, delete');

@itisnajim
Copy link

itisnajim commented Jan 5, 2023

I'm still having trouble setting up a nginx server block for a domain to work with realtime:v2.0.0. i get this error:

supabase-realtime  | 20:39:20.396 project=realtime-dev external_id=realtime-dev [error] Auth error: {:error, :signature_error}
supabase-kong      | 172.26.0.1 - - [05/Jan/2023:20:39:20 +0000] "GET /realtime/v1/websocket?apikey=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.ewogICAgInJvbGUiOiAiYW5vbiIsCiAgICAiaXNzIjogInN1cGFiYXNlIiwKICAgICJpYXQiOiAxNjcyODczMjAwLAogICAgImV4cCI6IDE4MzA2Mzk2MDAKfQ.BgAByr8s38H3uvO5CGt_ijULoyTYOjBi9JtIlXmz4vA&vsn=1.0.0 HTTP/1.1" 403 0 "-" "Dart/2.18 (dart:io)"

this is the payload used:
JWT: cor19x6wYudqK/HY8tKJOBoA0KD/zxM/SxxkI1zPOvSCs67x4q75+0yV07SWdm0T

{
  "role": "service_role",
  "iss": "Issuer",
  "nbf": 1610078130,
  "iat": 1672873200,
  "exp": 1830639600
}

The generated service role key:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoic2VydmljZV9yb2xlIiwiaXNzIjoiSXNzdWVyIiwibmJmIjoxNjEwMDc4MTMwLCJpYXQiOjE2NzI4NzMyMDAsImV4cCI6MTgzMDYzOTYwMH0.ttDzVV38_BskQhlr7BcVZ-ffuUKA3f5xC6UCOFoFRYo

If anyone has experience with this and could provide some guidance, it would be greatly appreciated. Thanks in advance for your help.

@itisnajim
Copy link

itisnajim commented Jan 6, 2023

i was stupid, the JWT should be a1d99c8b-91b6-47b2-8f3c-aa7d9a9ad20f (I thought the one in the .env is used, but maybe hard coded:

"jwt_secret" => "a1d99c8b-91b6-47b2-8f3c-aa7d9a9ad20f",
)
Now i have this error message is repeating on the terminal multiple times:

[error] Could not create schema migrations table.
supabase-realtime  |    {%Postgrex.Error{
supabase-realtime  |       message: nil,
supabase-realtime  |       postgres: %{
supabase-realtime  |         code: :insufficient_privilege,
supabase-realtime  |         file: "aclchk.c",
supabase-realtime  |         line: "3447",
supabase-realtime  |         message: "permission denied for schema realtime",
supabase-realtime  |         pg_code: "42501",
supabase-realtime  |         position: "28",
supabase-realtime  |         routine: "aclcheck_error",
supabase-realtime  |         severity: "ERROR",
supabase-realtime  |         unknown: "ERROR"
supabase-realtime  |       },
supabase-realtime  |       connection_id: 552,
supabase-realtime  |       query: nil
supabase-realtime  |     },

in the flutter console no error(s) shown:
flutter: SUBSCRIBED err: null

@w3b6x9
Copy link
Member

w3b6x9 commented Jan 6, 2023

Closing this issue b/c the cli and self-hosting (supabase/supabase#11457) Realtime versions are up to date.

@w3b6x9 w3b6x9 closed this as completed Jan 6, 2023
@w3b6x9 w3b6x9 unpinned this issue Jan 6, 2023
@itisnajim
Copy link

@bhvngt sorry to bother you again, i want to ask how did you get rid of this error:
[error] Could not create schema migrations table. ?

Now i have this error message is repeating on the terminal multiple times:

[error] Could not create schema migrations table.
supabase-realtime  |    {%Postgrex.Error{
supabase-realtime  |       message: nil,
supabase-realtime  |       postgres: %{
supabase-realtime  |         code: :insufficient_privilege,
supabase-realtime  |         file: "aclchk.c",
supabase-realtime  |         line: "3447",
supabase-realtime  |         message: "permission denied for schema realtime",
supabase-realtime  |         pg_code: "42501",
supabase-realtime  |         position: "28",
supabase-realtime  |         routine: "aclcheck_error",
supabase-realtime  |         severity: "ERROR",
supabase-realtime  |         unknown: "ERROR"
supabase-realtime  |       },
supabase-realtime  |       connection_id: 552,
supabase-realtime  |       query: nil
supabase-realtime  |     },

in the flutter console no error(s) shown: flutter: SUBSCRIBED err: null

@bhvngt
Copy link

bhvngt commented Jan 6, 2023

@itisnajim I never got this error. Are you using the same PG_USER to initialise db container as well as to connect to the realtime container?

@itisnajim
Copy link

@itisnajim I never got this error. Are you using the same PG_USER to initialise db container as well as to connect to the realtime container?

@bhvngt , its a fresh download, i guess = postgres according to:

https://github.com/supabase/supabase/blob/master/docker/docker-compose.yml#L117

https://github.com/supabase/supabase/blob/master/docker/.env.example#L18

i just created an issue in the main repo
supabase/supabase#11484

@bhvngt
Copy link

bhvngt commented Jan 6, 2023

@itisnajim I am not using the latest download. I had manually created realtime.sql earlier with following sql.

create schema if not exists _realtime;
create schema if not exists realtime;

create publication supabase_realtime with (publish = 'insert, update, delete');

Since POSTGRES_USER is not set as env variable inside db container, it could be the cause of the error.

Try setting POSTGRES_USER for the db container

@itisnajim
Copy link

itisnajim commented Jan 6, 2023

i solved by replacing ${POSTGRES_USER} with supabase_admin
in:

  realtime:
    container_name: realtime-dev.supabase-realtime
    image: supabase/realtime:v2.0.2
    ...
      DB_USER: ${POSTGRES_USER} # should be DB_USER: supabase_admin

code line:
https://github.com/supabase/supabase/blob/master/docker/docker-compose.yml#L117

now the docker logs filled with:

realtime-dev.supabase-realtime  | 13:47:51.016 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:00.546 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:09.600 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:16.805 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:23.356 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:31.382 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:41.215 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:48:51.012 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:49:00.391 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}
realtime-dev.supabase-realtime  | 13:49:06.924 project=realtime-dev external_id=realtime-dev [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}

@itisnajim
Copy link

restarting the docker and it shows no [error] Subscribing to PostgreSQL failed: {:error, {:subscription_insert_failed, %{"event" => "*", "schema" => "public"}}}

@jsmueller7
Copy link

jsmueller7 commented Jun 7, 2023

Hi @bhvngt

I also keep getting following warning

 06:01:34.137 [warning] [libcluster:fly6pn] dns polling strategy is selected, but query or basename param is invalid: %{node_basename: "realtime", query: ""}

Here too. Every 10 seconds I receive this warning twice.

[warning] [libcluster:fly6pn] dns polling strategy is selected, but query or basename param is invalid: %{node_basename: "realtime", query: nil}

I am using a newer version of realtime (2.13.1) and I haven't actually checkt, if these messages are of any relevance. Did they go away when you changed your config? Are they related to a misconfiguration at all? Has anyone else has them?

psyrenpark added a commit to psyrenpark/supabase-on-aws that referenced this issue Jan 26, 2024
This commit addresses two key issues encountered while using the Supabase library in a self-hosted environment on AWS.

1. Realtime Service Not Functioning:
   - Identified Issue: Realtime service was non-operational, as discussed in supabase/realtime#372.
   - Resolution: Upgraded the Realtime Docker image from 'public.ecr.aws/supabase/realtime:v2.25.27' to 'public.ecr.aws/supabase/realtime:v2.25.60', resolving the issue.

2. StudioBranch Error:
   - Identified Issue: Updating StudioBranch from 'v0.23.09' to 'v0.23.11' resulted in an AWS Amplify CustomerError related to reading 'next' version in package.json (https://stackoverflow.com/questions/76296605/aws-amplify-customererror-cannot-read-next-version-in-package-json).
   - Temporary Solution: Reverted to version 'v0.23.09' and pushed via CodeCommit, which led to successful redeployment.

This commit should enhance the stability and functionality of the Supabase library in similar self-hosted setups. Further investigation into the StudioBranch error is suggested for a more permanent solution.
@AntonOfTheWoods
Copy link

@jsmueller7 did those warnings turn out to symptomatic of deeper problems or are they "normal" errors?

@jsmueller7
Copy link

Hi @AntonOfTheWoods ,
these warnings were due to the fact that I was running Supabase in a cluster and the env DNS_NODES was not configured. So my post has been resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

9 participants