Send events through postgres notify

[saneery]

#2354

Realtime events are sending by Registry, but when indexer runs separately from webapp, on different instances, events aren't delivered.

So I updated publisher to send events through Postgres notification and added Listener that listen to events from Postgres and send them to subscribed processes

To test you can run indexer and webapp separately and make sure that realtime updates are delivered in the main page.

$ mix cmd --app indexer iex -S mix
$ mix cmd --app block_scout_web mix phx.server

[coveralls]

Pull Request Test Coverage Report for Build 3bb441bb-ca24-467d-a26e-7e2f2216da9f

• 7 of 22 (31.82%) changed or added relevant lines in 4 files are covered.
• 4 unchanged lines in 2 files lost coverage.
• Overall coverage decreased (-0.2%) to 76.597%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
apps/explorer/lib/explorer/chain/events/db_sender.ex	0	6	0.0%
apps/explorer/lib/explorer/chain/events/listener.ex	0	9	0.0%

Files with Coverage Reduction	New Missed Lines	%
apps/block_scout_web/lib/block_scout_web/controllers/chain/market_history_chart_controller.ex	2	71.43%
apps/indexer/lib/indexer/fetcher/replaced_transaction.ex	2	90.91%

Totals
Change from base Build 2afb100c-ab3c-44d0-9d7d-8b6d646b7cfe:	-0.2%
Covered Lines:	5289
Relevant Lines:	6905

---

💛 - Coveralls

[vbaranov]

@saneery

• please update the corresponding chapter with the new env vars
• please add those env var(s) to the description of PR
• can we make sending events through postgres notify configurable via env var and if it is not set - use the current websocket architecture?

[saneery]

@vbaranov

• please update the corresponding chapter with the new env vars

I didn't add new vars

• can we make sending events through postgres notify configurable via env var and if it is not set - use the current websocket architecture?

Yes, I think it's possible

[vbaranov]

I didn't add new vars

Ah, indeed, sorry. I mistakenly thought, that HEALTHY_BLOCKS_PERIOD introduced in this PR.

[jcortejoso]

Hi! Is there any plan to merge/review this PR? I have been using Blockscout from this commit and so far so good.

[jcortejoso]

I have been testing this PR and I think it breaks the option to configure the database with the env variable DATABASE_URL. When running from this branch and configuring database access with DATABASE_URL I get the next error:

╰─ mix phx.server
2019-10-01T22:10:35.339 application=que [info] [Que] Booting Que
2019-10-01T22:10:35.376 application=que [info] [Que] Booting Server Supervisor for Workers
2019-10-01T22:10:35.496 [info] Application explorer exited: Explorer.Application.start(:normal, []) returned an error: shutdown: failed to start child: Explorer.Chain.Events.Listener
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, {%KeyError{key: :database, message: nil, term: [types: nil, types: Postgrex.DefaultTypes, hostname: "localhost", username: "jcortejoso", sync_connect: true, migration_timestamps: [type: :utc_datetime_usec], url: "postgres://postgres:1234@pucela.local:5432/blockscout?ssl=false", pool_size: 20, ssl: false, prepare: :unnamed, timeout: 60000]}, [{Keyword, :fetch!, 2, [file: 'lib/keyword.ex', line: 393]}, {Postgrex.Protocol, :startup, 2, [file: 'lib/postgrex/protocol.ex', line: 676]}, {Postgrex.Protocol, :handshake, 2, [file: 'lib/postgrex/protocol.ex', line: 579]}, {Postgrex.Notifications, :connect, 2, [file: 'lib/postgrex/notifications.ex', line: 153]}, {Postgrex.Notifications, :init, 1, [file: 'lib/postgrex/notifications.ex', line: 143]}, {Connection, :init_it, 6, [file: 'lib/connection.ex', line: 424]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
            (explorer) lib/explorer/chain/events/listener.ex:15: Explorer.Chain.Events.Listener.init/1
            (stdlib) gen_server.erl:374: :gen_server.init_it/2
            (stdlib) gen_server.erl:342: :gen_server.init_it/6
            (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
** (Mix) Could not start application explorer: Explorer.Application.start(:normal, []) returned an error: shutdown: failed to start child: Explorer.Chain.Events.Listener
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, {%KeyError{key: :database, message: nil, term: [types: nil, types: Postgrex.DefaultTypes, hostname: "localhost", username: "jcortejoso", sync_connect: true, migration_timestamps: [type: :utc_datetime_usec], url: "postgres://postgres:1234@pucela.local:5432/blockscout?ssl=false", pool_size: 20, ssl: false, prepare: :unnamed, timeout: 60000]}, [{Keyword, :fetch!, 2, [file: 'lib/keyword.ex', line: 393]}, {Postgrex.Protocol, :startup, 2, [file: 'lib/postgrex/protocol.ex', line: 676]}, {Postgrex.Protocol, :handshake, 2, [file: 'lib/postgrex/protocol.ex', line: 579]}, {Postgrex.Notifications, :connect, 2, [file: 'lib/postgrex/notifications.ex', line: 153]}, {Postgrex.Notifications, :init, 1, [file: 'lib/postgrex/notifications.ex', line: 143]}, {Connection, :init_it, 6, [file: 'lib/connection.ex', line: 424]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 249]}]}}
            (explorer) lib/explorer/chain/events/listener.ex:15: Explorer.Chain.Events.Listener.init/1
            (stdlib) gen_server.erl:374: :gen_server.init_it/2
            (stdlib) gen_server.erl:342: :gen_server.init_it/6
            (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

Configuration using config files work well.

[ayrat555]

@jcortejoso I'll finish this. Thanks for testing

[jcortejoso]

@ayrat555 thanks. I added this to apps/explorer/config/prod.exs and seems to work now:

diff --git a/apps/explorer/config/prod.exs b/apps/explorer/config/prod.exs
index 0957ffa1f..1e040434a 100644
--- a/apps/explorer/config/prod.exs
+++ b/apps/explorer/config/prod.exs
@@ -2,7 +2,12 @@ use Mix.Config

 # Configures the database
 config :explorer, Explorer.Repo,
-  url: System.get_env("DATABASE_URL"),
+  url: System.get_env("DATABASE_URL") || "postgresql://postgres:1234@localhost:5432/blockscout",
+  username: System.get_env("DATABASE_USER") || "postgres",
+  password: System.get_env("DATABASE_PASSWORD") || "1234",
+  database: System.get_env("DATABASE_DB") || "blockscout",
+  hostname: System.get_env("DATABASE_HOSTNAME") || "localhost",
+  port: System.get_env("DATABASE_PORT") || "5432",
   pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10"),
   ssl: String.equivalent?(System.get_env("ECTO_USE_SSL") || "true", "true"),
   prepare: :unnamed,

Best!

[ayrat555]

this kind of variables should be configured with prod.secret.exs file

[pasqu4le]

@ayrat555 there is no changelog entry