IEx 1.3.0 - Postgrex.Protocol failed to connect :econnreset

[nicbet]

Once in a while, execution of iex -S mix will result in one or more error lines stating:

❯ iex -S mix                                                                                                                  
Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Interactive Elixir (1.3.0) - press Ctrl+C to exit (type h() ENTER for help)

iex(1)> [error] Postgrex.Protocol (#PID<0.210.0>) failed to connect: ** (Postgrex.Error) tcp connect: connection reset by peer - :econnreset
iex(1)> [error] Postgrex.Protocol (#PID<0.210.0>) failed to connect: ** (Postgrex.Error) tcp connect: connection reset by peer - :econnreset
iex(1)> [error] Postgrex.Protocol (#PID<0.210.0>) failed to connect: ** (Postgrex.Error) tcp connect: connection reset by peer - :econnreset

Relevant dependency versions are listed as:

* connection 1.0.3 (Hex package) (mix)
  locked at 1.0.3 (connection) 3145f741

* poolboy 1.5.1 (Hex package) (rebar)
  locked at 1.5.1 (poolboy) 6b461639

* db_connection 1.0.0-rc.3 (Hex package) (mix)
  locked at 1.0.0-rc.3 (db_connection) d9ceb670

* phoenix 1.2.0 (Hex package) (mix)
  locked at 1.2.0 (phoenix) 1bdeb99c

* postgrex 0.11.2 (Hex package) (mix)
  locked at 0.11.2 (postgrex) 139755c1

* ecto 2.0.1 (Hex package) (mix)
  locked at 2.0.1 (ecto) cf97a4d3

* phoenix_ecto 3.0.0 (Hex package) (mix)
  locked at 3.0.0 (phoenix_ecto) b947aaf0

[fishcakez]

@nicbet I am afraid I don't think there is enough information to go on here. Would need to at least see the configuration and some tips how to reproduce.

[nicbet]

@fishcakez Configuration is what Phoenix 1.2.0 generates out-of-the-box via mix phoenix new. I'll be happy to post a copy if needed.

[fishcakez]

Please because I have no idea :)

[nicbet]

Application generated via mix phoenix new blub

File mix.exs:

defmodule Blub.Mixfile do
  use Mix.Project

  def project do
    [app: :blub,
     version: "0.0.1",
     elixir: "~> 1.2",
     elixirc_paths: elixirc_paths(Mix.env),
     compilers: [:phoenix, :gettext] ++ Mix.compilers,
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     aliases: aliases(),
     deps: deps()]
  end

  # Configuration for the OTP application.
  #
  # Type `mix help compile.app` for more information.
  def application do
    [mod: {Blub, []},
     applications: [:phoenix, :phoenix_pubsub, :phoenix_html, :cowboy, :logger, :gettext,
                    :phoenix_ecto, :postgrex]]
  end

  # Specifies which paths to compile per environment.
  defp elixirc_paths(:test), do: ["lib", "web", "test/support"]
  defp elixirc_paths(_),     do: ["lib", "web"]

  # Specifies your project dependencies.
  #
  # Type `mix help deps` for examples and options.
  defp deps do
    [{:phoenix, "~> 1.2.0"},
     {:phoenix_pubsub, "~> 1.0"},
     {:phoenix_ecto, "~> 3.0"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_html, "~> 2.6"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:gettext, "~> 0.11"},
     {:cowboy, "~> 1.0"}]
  end

  # Aliases are shortcuts or tasks specific to the current project.
  # For example, to create, migrate and run the seeds file at once:
  #
  #     $ mix ecto.setup
  #
  # See the documentation for `Mix` for more info on aliases.
  defp aliases do
    ["ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
     "ecto.reset": ["ecto.drop", "ecto.setup"],
     "test": ["ecto.create --quiet", "ecto.migrate", "test"]]
  end
end

File config/config.exs

# This file is responsible for configuring your application
# and its dependencies with the aid of the Mix.Config module.
#
# This configuration file is loaded before any dependency and
# is restricted to this project.
use Mix.Config

# General application configuration
config :blub,
  ecto_repos: [Blub.Repo]

# Configures the endpoint
config :blub, Blub.Endpoint,
  url: [host: "localhost"],
  secret_key_base: "MQAsDf6am6tf++3I20DZ72khMo23Ea+hB9bSRhOGnPvFcQ6F/qbz9ec/+19NaJm0",
  render_errors: [view: Blub.ErrorView, accepts: ~w(html json)],
  pubsub: [name: Blub.PubSub,
           adapter: Phoenix.PubSub.PG2]

# Configures Elixir's Logger
config :logger, :console,
  format: "$time $metadata[$level] $message\n",
  metadata: [:request_id]

# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{Mix.env}.exs"

File config/dev.exs:

use Mix.Config

# For development, we disable any cache and enable
# debugging and code reloading.
#
# The watchers configuration can be used to run external
# watchers to your application. For example, we use it
# with brunch.io to recompile .js and .css sources.
config :blub, Blub.Endpoint,
  http: [port: 4000],
  debug_errors: true,
  code_reloader: true,
  check_origin: false,
  watchers: [node: ["node_modules/brunch/bin/brunch", "watch", "--stdin",
                    cd: Path.expand("../", __DIR__)]]


# Watch static and templates for browser reloading.
config :blub, Blub.Endpoint,
  live_reload: [
    patterns: [
      ~r{priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$},
      ~r{priv/gettext/.*(po)$},
      ~r{web/views/.*(ex)$},
      ~r{web/templates/.*(eex)$}
    ]
  ]

# Do not include metadata nor timestamps in development logs
config :logger, :console, format: "[$level] $message\n"

# Set a higher stacktrace during development. Avoid configuring such
# in production as building large stacktraces may be expensive.
config :phoenix, :stacktrace_depth, 20

# Configure your database
config :blub, Blub.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "postgres",
  password: "postgres",
  database: "blub_dev",
  hostname: "localhost",
  pool_size: 10

[ericmj]

Unless you can give us something more to go on there's not much we can do. You are the first one reporting this so my guess would be something with the local setup.

Please let us know if you have any more information.

[andykingking]

I am also experiencing this (both locally and on CI). Running the white_bread.run mix task sporadically produces the above error.

Postgres v9.4 is running in Docker using the official image, with the standard port exposed to local.

Elixir version: 1.3.1
Postgrex version: 0.11.2

File mix.lock

%{"bunt": {:hex, :bunt, "0.1.6", "5d95a6882f73f3b9969fdfd1953798046664e6f77ec4e486e6fafc7caad97c6f", [:mix], []},
  "certifi": {:hex, :certifi, "0.4.0", "a7966efb868b179023618d29a407548f70c52466bf1849b9e8ebd0e34b7ea11f", [:rebar3], []},
  "connection": {:hex, :connection, "1.0.3", "3145f7416be3df248a4935f24e3221dc467c1e3a158d62015b35bd54da365786", [:mix], []},
  "cowboy": {:hex, :cowboy, "1.0.4", "a324a8df9f2316c833a470d918aaf73ae894278b8aa6226ce7a9bf699388f878", [:rebar, :make], [{:cowlib, "~> 1.0.0", [hex: :cowlib, optional: false]}, {:ranch, "~> 1.0", [hex: :ranch, optional: false]}]},
  "cowlib": {:hex, :cowlib, "1.0.2", "9d769a1d062c9c3ac753096f868ca121e2730b9a377de23dec0f7e08b1df84ee", [:make], []},
  "credo": {:hex, :credo, "0.4.5", "5c5daaf50a2a96068c0f21b6fbd382d206702efa8836a946eeab0b8ac25f5f22", [:mix], [{:bunt, "~> 0.1.6", [hex: :bunt, optional: false]}]},
  "db_connection": {:hex, :db_connection, "1.0.0-rc.3", "d9ceb670fe300271140af46d357b669983cd16bc0d01206d7d3222dde56cf038", [:mix], [{:sbroker, "~> 1.0.0-beta.3", [hex: :sbroker, optional: true]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: true]}, {:connection, "~> 1.0.2", [hex: :connection, optional: false]}]},
  "decimal": {:hex, :decimal, "1.1.2", "79a769d4657b2d537b51ef3c02d29ab7141d2b486b516c109642d453ee08e00c", [:mix], []},
  "ecto": {:hex, :ecto, "2.0.2", "b02331c1f20bbe944dbd33c8ecd8f1ccffecc02e344c4471a891baf3a25f5406", [:mix], [{:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: true]}, {:sbroker, "~> 1.0-beta", [hex: :sbroker, optional: true]}, {:mariaex, "~> 0.7.7", [hex: :mariaex, optional: true]}, {:postgrex, "~> 0.11.2", [hex: :postgrex, optional: true]}, {:db_connection, "~> 1.0-rc.2", [hex: :db_connection, optional: true]}, {:decimal, "~> 1.0", [hex: :decimal, optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: false]}]},
  "espec": {:hex, :espec, "0.8.22", "65712cf269071202b6196a91eb752d954841da2c3452726b56344be8b78260d2", [:mix], [{:meck, "~> 0.8.4", [hex: :meck, optional: false]}]},
  "espec_phoenix": {:git, "https://github.com/diogobeda/espec_phoenix.git", "3ceeece68d72e727e406ddcb87da42abac7f5ace", []},
  "excoveralls": {:hex, :excoveralls, "0.5.5", "d97b6fc7aa59c5f04f2fa7ec40fc0b7555ceea2a5f7e7c442aad98ddd7f79002", [:mix], [{:hackney, ">= 0.12.0", [hex: :hackney, optional: false]}, {:exjsx, "~> 3.0", [hex: :exjsx, optional: false]}]},
  "exjsx": {:hex, :exjsx, "3.2.0", "7136cc739ace295fc74c378f33699e5145bead4fdc1b4799822d0287489136fb", [:mix], [{:jsx, "~> 2.6.2", [hex: :jsx, optional: false]}]},
  "floki": {:hex, :floki, "0.9.0", "e952ca71a453f7827ab5405106ac8d9ac5c9602d18aa5d2d893e5b9944e2499e", [:mix], [{:mochiweb_html, "~> 2.15", [hex: :mochiweb_html, optional: false]}]},
  "gettext": {:hex, :gettext, "0.11.0", "80c1dd42d270482418fa158ec5ba073d2980e3718bacad86f3d4ad71d5667679", [:mix], []},
  "hackney": {:hex, :hackney, "1.6.0", "8d1e9440c9edf23bf5e5e2fe0c71de03eb265103b72901337394c840eec679ac", [:rebar3], [{:ssl_verify_fun, "1.1.0", [hex: :ssl_verify_fun, optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, optional: false]}, {:metrics, "1.0.1", [hex: :metrics, optional: false]}, {:idna, "1.2.0", [hex: :idna, optional: false]}, {:certifi, "0.4.0", [hex: :certifi, optional: false]}]},
  "httpoison": {:hex, :httpoison, "0.9.0", "68187a2daddfabbe7ca8f7d75ef227f89f0e1507f7eecb67e4536b3c516faddb", [:mix], [{:hackney, "~> 1.6.0", [hex: :hackney, optional: false]}]},
  "idna": {:hex, :idna, "1.2.0", "ac62ee99da068f43c50dc69acf700e03a62a348360126260e87f2b54eced86b2", [:rebar3], []},
  "jsx": {:hex, :jsx, "2.6.2", "213721e058da0587a4bce3cc8a00ff6684ced229c8f9223245c6ff2c88fbaa5a", [:mix, :rebar], []},
  "meck": {:hex, :meck, "0.8.4", "59ca1cd971372aa223138efcf9b29475bde299e1953046a0c727184790ab1520", [:rebar, :make], []},
  "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], []},
  "mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], []},
  "mochiweb_html": {:hex, :mochiweb_html, "2.15.0", "d7402e967d7f9f2912f8befa813c37be62d5eeeddbbcb6fe986c44e01460d497", [:rebar3], []},
  "phoenix": {:hex, :phoenix, "1.2.0", "1bdeb99c254f4c534cdf98fd201dede682297ccc62fcac5d57a2627c3b6681fb", [:mix], [{:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: false]}, {:phoenix_pubsub, "~> 1.0", [hex: :phoenix_pubsub, optional: false]}, {:plug, "~> 1.1", [hex: :plug, optional: false]}, {:cowboy, "~> 1.0", [hex: :cowboy, optional: true]}]},
  "phoenix_ecto": {:hex, :phoenix_ecto, "3.0.0", "b947aaf03d076f5b1448f87828f22fb7710478ee38455c67cc3fe8e9a4dfd015", [:mix], [{:ecto, "~> 2.0.0-rc", [hex: :ecto, optional: false]}, {:phoenix_html, "~> 2.6", [hex: :phoenix_html, optional: true]}]},
  "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.0.0", "c31af4be22afeeebfaf246592778c8c840e5a1ddc7ca87610c41ccfb160c2c57", [:mix], []},
  "plug": {:hex, :plug, "1.1.6", "8927e4028433fcb859e000b9389ee9c37c80eb28378eeeea31b0273350bf668b", [:mix], [{:cowboy, "~> 1.0", [hex: :cowboy, optional: true]}]},
  "poison": {:hex, :poison, "2.2.0", "4763b69a8a77bd77d26f477d196428b741261a761257ff1cf92753a0d4d24a63", [:mix], []},
  "poolboy": {:hex, :poolboy, "1.5.1", "6b46163901cfd0a1b43d692657ed9d7e599853b3b21b95ae5ae0a777cf9b6ca8", [:rebar], []},
  "postgrex": {:hex, :postgrex, "0.11.2", "139755c1359d3c5c6d6e8b1ea72556d39e2746f61c6ddfb442813c91f53487e8", [:mix], [{:connection, "~> 1.0", [hex: :connection, optional: false]}, {:db_connection, "~> 1.0-rc", [hex: :db_connection, optional: false]}, {:decimal, "~> 1.0", [hex: :decimal, optional: false]}]},
  "ranch": {:hex, :ranch, "1.2.1", "a6fb992c10f2187b46ffd17ce398ddf8a54f691b81768f9ef5f461ea7e28c762", [:make], []},
  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.0", "edee20847c42e379bf91261db474ffbe373f8acb56e9079acb6038d4e0bf414f", [:rebar, :make], []},
  "white_bread": {:hex, :white_bread, "2.7.0", "8c5cb130effc64b1814cdf187abd1047b87cebd7bb4c154aa7f75649a69dd862", [:mix], []}}```

[nicbet]

@andykingking Like you I am executing my PSQL as a Docker container, official image (version 9.5.3). I'm wondering if some configuration in that docker image is messing with the connection pool?

setting config :app, App.Repo, pool_size: 2 usually makes the problem less frequent. Maybe the OTP app has problems closing/releasing connections to PSQL in/from the connection pool, as the PIDs in the error lines correspond directly to the con pool. So if the PSQL docker container is configured for max_connections=100 and postgrex is configured with a pool_size of 10, 10 fast subsequent restarts of the app would exhaust the max_connections and PSQL throws econn_reset. That's consistent with the behaviour I've been observing.

[andykingking]

Just to clarify, the Postgres instance that I am connecting to is in a Docker container, however my Elixir app is running locally, which happens to have Postgres installed.

[josevalim]

@nicbet in other words, if you keep the pool size as 10 but increase max_connections on PSQL side to 1000, the problem disappears?

[fishcakez]

Postgresql will return a postgresql protocol fatal error on too many connections. I am pretty sure this is an issue with your docker configuration but I am not sure what.

[andykingking]

@fishcakez Here is my docker-compose file, I'm using the official docker image. What do you think it could be?

database:
  image: postgres:9.5
  environment:
    - POSTGRES_PASSWORD=postgres
    - POSTGRES_DB=database_name
  ports:
    - 5432:5432

[fishcakez]

@andykingking sorry I dont know enough about docker to know. If the max connections is reached at the database level you should get log messages like this with postgresql 9.5:

failed to connect: ** (Postgrex.Error) FATAL (too_many_connections): sorry, too many clients already

[andykingking]

@fishcakez Thus far I haven't seen the error you mentioned. I'll investigate further.

[fbessez]

Did you ever figure this out? I am having the same issue. I too am trying to dockerize my phoenix app and this is occurring on my compose up command.

[homanchou]

I'm getting something similar while going through the Phoenix book and doing the Rumbl application. I'm not trying to dockerize or anything. I'm on the chapter with tests and getting these errors which I'm not sure if this is related, but I can't figure it out.

mix test
22:46:37.169 [error] Postgrex.Protocol (#PID<0.206.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
22:46:37.169 [error] Postgrex.Protocol (#PID<0.203.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
22:46:37.169 [error] Postgrex.Protocol (#PID<0.205.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
22:46:37.169 [error] Postgrex.Protocol (#PID<0.207.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
22:46:37.169 [error] Postgrex.Protocol (#PID<0.208.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
22:46:37.169 [error] Postgrex.Protocol (#PID<0.204.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
....

  1) test render any other (Rumbl.ErrorViewTest)
     test/views/error_view_test.exs:17
     ** (MatchError) no match of right hand side value: {:error, %DBConnection.ConnectionError{message: "connection not available because of disconnection"}}
     stacktrace:
       (rumbl) test/support/conn_case.ex:37: Rumbl.ConnCase.__ex_unit_setup_0/1
       (rumbl) test/support/conn_case.ex:1: Rumbl.ConnCase.__ex_unit__/2
       test/views/error_view_test.exs:1: Rumbl.ErrorViewTest.__ex_unit__/2

....

Finished in 0.4 seconds
26 tests, 14 failures

Tests were humming along before... I don't know what I did, seems like something got corrupted. Even if I git checkout older commits that had all tests passing before and run mix clean then mix test I still get the errors.

I've pushed up what I have to a repo here: https://github.com/homanchou/phoenix_book_exercises

I also tried git cloning what I pushed up and running mix test in that new directory but that also has the same errors.

elixir --version
Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.1

UPDATE: Completely rebooting my mac seems to have fixed the issue. Now all tests pass.

[fishcakez]

@homanchou the error you mention is different. See tcp recv: closed, which likely means handshaking timed out - we can improve that error. Whereas the above errors have tcp connect: ....

[aisrael]

Just to chime in, getting the same error as @homanchou, but I've observed an interesting chain of events, after leaving my Phoenix app alone for some time:

10:18:24 [info] Running HelloPhoenix.Endpoint with Cowboy using http://localhost:4000
10:18:27 10 Aug 10:18:27 - info: compiled 6 files into 2 files, copied 3 in 2.9 sec
13:16:58 [error] Postgrex.Protocol (#PID<0.231.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.233.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.224.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.228.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.226.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.225.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.230.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.229.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.227.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
13:16:58 [error] Postgrex.Protocol (#PID<0.232.0>) disconnected: ** (Postgrex.Error) tcp recv: timeout
14:03:23 [error] Postgrex.Protocol (#PID<0.230.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
14:03:23 [error] Postgrex.Protocol (#PID<0.227.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
14:03:23 [error] Postgrex.Protocol (#PID<0.232.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed
14:03:23 [error] Postgrex.Protocol (#PID<0.229.0>) failed to connect: ** (Postgrex.Error) tcp recv: closed

(And so on...)

My environment is similar to @nicbet. This is a 'stock' Phoenix app (mix phoenix.new hello_phoenix, following the Phoenix tutorials) running on:

• Elixir v1.3.2
• postgrex v0.11.2
• Postgres 9.5, running under Docker v1.12.0, with port 5432 exposed on local

(Edited to add...)

• OS X 10.11.6 if that matters

[fishcakez]

Are all of these on OSX? I haven't been able to reproduce this on linux.

[nicbet]

@fishcakez my environment is:

• Mac OS X 10.11.6
• Docker 1.12.0-beta22 (build: 11222).
• Elixir 1.3.2
• Erlang/OTP 19 [erts-8.0.2]
• Official postgres:latest docker image(s)

So very much the same as aisrael.

[matteodepalo]

I'm experiencing the same issue as @nicbet. I've installed postgres using homebrew and tried to increase the number of connections to 1000, but the result is the same: when I start the Phoenix server I occasionally get:

[info] Running KnockoutApi.Endpoint with Cowboy using http://localhost:4000
[error] Postgrex.Protocol (#PID<0.808.0>) failed to connect: ** (Postgrex.Error) tcp connect: connection refused - :econnrefused
[error] Postgrex.Protocol (#PID<0.808.0>) failed to connect: ** (Postgrex.Error) tcp connect: connection refused - :econnrefused
...

To give some context, I've recently updated a Phoenix 1.1.x app to 1.2.0 with Ecto 2.0.4. After the update everything was working fine, but then I reset the db by dropping, creating, and migrating and I started getting this kind of errors.

Sorry if this not enough to reproduce, I'm hoping this can give more info around this issue and I'm curious to know if others have received the same error from their phoenix apps.

[joshcrews]

I got this error to when I went from a
Ecto 1 -> 2
Phoenix 1.1 -> 1.2
Elixir 1.2 -> 1.3
migration at the same time on heroku

The prod.exs file has a pool_size of 20, which is also the limit of Postgres connections on the hobby:dev size plan.

When I'd do anything else that run the single web dyno (like heroku run mix ecto.migrate or heroku run iex -S mix) the console runs 2 or 3 of these error warnings a second: (Postgrex.Error) FATAL (too_many_connections)

I just dropped my pool_size from 20 to 10, and the errors are gone.

I do not know why this never happened when I was running Ecto 1, Phoenix 1.1, Elixir 1.2 on the same heroku dyno size before the migration.

[fishcakez]

@joshcrews In Ecto 1.0 connections were setup lazily so if your app didnt need 20 connections only a subset would connect. However in Ecto 2.0 connections are setup eagerly, which should provide better behaviour when errors occur. If you are getting too_many_connections (opposed to econnrefused) then this would be expected here.

The following PR phoenixframework/phoenix_guides#513 and associated issue should provide you with some useful information given you are using phoenix and heroku.

[paulo-roberto-a]

I have this same problem

[fishcakez]

@nenoalmeida could you let us know the postgrex, postgresql, OS and elixir versions?

[samueleaton]

I am having the same problem. Here is a repo I just uploaded with all of my current code (its basically just the templates that phoenix produces). My entire console output is in the readme.

I also added the prod.secret.exs file to git to see if it helps. Please look over my Dockerfile and my docker-compose.yml file for elixir and postgres versions.

There is nothing sensitive in my code yet. I will most likely rebuild from scratch after I find out what the problem is.

OS is OSX El Cap 10.11.5 and I'm using Docker for Mac

[nateless]

Guys, having the same issue. It worked fine for several days, then at some point code raised an exception and after we applied a fix and recompile it, it throws this error. We didn't update any packages.

mix.lock

%{"apex": {:hex, :apex, "0.5.2", "bc4d44c3f824d70cd295fc2fd395243df34a1c1659f5ef17381f5fbee1c7f00e", [:mix], []},
  "certifi": {:hex, :certifi, "0.4.0", "a7966efb868b179023618d29a407548f70c52466bf1849b9e8ebd0e34b7ea11f", [:rebar3], []},
  "comeonin": {:hex, :comeonin, "2.5.3", "ccd70ebf465eaf4e11fc5a13fd0f5be2538b6b4975d4f7a13d571670b31da060", [:mix, :make, :make], []},
  "connection": {:hex, :connection, "1.0.4", "a1cae72211f0eef17705aaededacac3eb30e6625b04a6117c1b2db6ace7d5976", [:mix], []},
  "cowboy": {:hex, :cowboy, "1.0.4", "a324a8df9f2316c833a470d918aaf73ae894278b8aa6226ce7a9bf699388f878", [:rebar, :make], [{:cowlib, "~> 1.0.0", [hex: :cowlib, optional: false]}, {:ranch, "~> 1.0", [hex: :ranch, optional: false]}]},
  "cowlib": {:hex, :cowlib, "1.0.2", "9d769a1d062c9c3ac753096f868ca121e2730b9a377de23dec0f7e08b1df84ee", [:make], []},
  "db_connection": {:hex, :db_connection, "1.0.0-rc.5", "1d9ab6e01387bdf2de7a16c56866971f7c2f75aea7c69cae2a0346e4b537ae0d", [:mix], [{:connection, "~> 1.0.2", [hex: :connection, optional: false]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: true]}, {:sbroker, "~> 1.0.0-beta.3", [hex: :sbroker, optional: true]}]},
  "decimal": {:hex, :decimal, "1.1.2", "79a769d4657b2d537b51ef3c02d29ab7141d2b486b516c109642d453ee08e00c", [:mix], []},
  "ecto": {:hex, :ecto, "2.0.5", "7f4c79ac41ffba1a4c032b69d7045489f0069c256de606523c65d9f8188e502d", [:mix], [{:db_connection, "~> 1.0-rc.4", [hex: :db_connection, optional: true]}, {:decimal, "~> 1.1.2 or ~> 1.2", [hex: :decimal, optional: false]}, {:mariaex, "~> 0.7.7", [hex: :mariaex, optional: true]}, {:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: true]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: false]}, {:postgrex, "~> 0.12.0", [hex: :postgrex, optional: true]}, {:sbroker, "~> 1.0-beta", [hex: :sbroker, optional: true]}]},
  "fs": {:hex, :fs, "0.9.2", "ed17036c26c3f70ac49781ed9220a50c36775c6ca2cf8182d123b6566e49ec59", [:rebar], []},
  "gettext": {:hex, :gettext, "0.11.0", "80c1dd42d270482418fa158ec5ba073d2980e3718bacad86f3d4ad71d5667679", [:mix], []},
  "hackney": {:hex, :hackney, "1.6.1", "ddd22d42db2b50e6a155439c8811b8f6df61a4395de10509714ad2751c6da817", [:rebar3], [{:certifi, "0.4.0", [hex: :certifi, optional: false]}, {:idna, "1.2.0", [hex: :idna, optional: false]}, {:metrics, "1.0.1", [hex: :metrics, optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, optional: false]}, {:ssl_verify_fun, "1.1.0", [hex: :ssl_verify_fun, optional: false]}]},
  "httpoison": {:hex, :httpoison, "0.9.1", "6c2b4eaf2588a6f3ef29663d28c992531ca3f0bc832a97e0359bc822978e1c5d", [:mix], [{:hackney, "~> 1.6.0", [hex: :hackney, optional: false]}]},
  "idna": {:hex, :idna, "1.2.0", "ac62ee99da068f43c50dc69acf700e03a62a348360126260e87f2b54eced86b2", [:rebar3], []},
  "logger_file_backend": {:hex, :logger_file_backend, "0.0.9", "5c2f7d4a28431e695cdf94d191523dbafe609321a67bb654254897f546c393db", [:mix], []},
  "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], []},
  "mime": {:hex, :mime, "1.0.1", "05c393850524767d13a53627df71beeebb016205eb43bfbd92d14d24ec7a1b51", [:mix], []},
  "mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], []},
  "phoenix": {:hex, :phoenix, "1.2.1", "6dc592249ab73c67575769765b66ad164ad25d83defa3492dc6ae269bd2a68ab", [:mix], [{:cowboy, "~> 1.0", [hex: :cowboy, optional: true]}, {:phoenix_pubsub, "~> 1.0", [hex: :phoenix_pubsub, optional: false]}, {:plug, "~> 1.1", [hex: :plug, optional: false]}, {:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: false]}]},
  "phoenix_ecto": {:hex, :phoenix_ecto, "3.0.1", "42eb486ef732cf209d0a353e791806721f33ff40beab0a86f02070a5649ed00a", [:mix], [{:ecto, "~> 2.0", [hex: :ecto, optional: false]}, {:phoenix_html, "~> 2.6", [hex: :phoenix_html, optional: true]}, {:plug, "~> 1.0", [hex: :plug, optional: false]}]},
  "phoenix_html": {:hex, :phoenix_html, "2.6.2", "944a5e581b0d899e4f4c838a69503ebd05300fe35ba228a74439e6253e10e0c0", [:mix], [{:plug, "~> 1.0", [hex: :plug, optional: false]}]},
  "phoenix_live_reload": {:hex, :phoenix_live_reload, "1.0.5", "829218c4152ba1e9848e2bf8e161fcde6b4ec679a516259442561d21fde68d0b", [:mix], [{:fs, "~> 0.9.1", [hex: :fs, optional: false]}, {:phoenix, "~> 1.0 or ~> 1.2-rc", [hex: :phoenix, optional: false]}]},
  "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.0.0", "c31af4be22afeeebfaf246592778c8c840e5a1ddc7ca87610c41ccfb160c2c57", [:mix], []},
  "plug": {:hex, :plug, "1.2.0", "496bef96634a49d7803ab2671482f0c5ce9ce0b7b9bc25bc0ae8e09859dd2004", [:mix], [{:cowboy, "~> 1.0", [hex: :cowboy, optional: true]}, {:mime, "~> 1.0", [hex: :mime, optional: false]}]},
  "poison": {:hex, :poison, "2.2.0", "4763b69a8a77bd77d26f477d196428b741261a761257ff1cf92753a0d4d24a63", [:mix], []},
  "poolboy": {:hex, :poolboy, "1.5.1", "6b46163901cfd0a1b43d692657ed9d7e599853b3b21b95ae5ae0a777cf9b6ca8", [:rebar], []},
  "postgrex": {:hex, :postgrex, "0.12.0", "bdeeb4c42768c47c3c92228e66c70357fe9a9384fbc9de06abba774b22dd0635", [:mix], [{:connection, "~> 1.0", [hex: :connection, optional: false]}, {:db_connection, "~> 1.0-rc.4", [hex: :db_connection, optional: false]}, {:decimal, "~> 1.0", [hex: :decimal, optional: false]}]},
  "ranch": {:hex, :ranch, "1.2.1", "a6fb992c10f2187b46ffd17ce398ddf8a54f691b81768f9ef5f461ea7e28c762", [:make], []},
  "redix": {:hex, :redix, "0.4.0", "c8cfad755252e5441c4119437ba3a8989518af7aa90dbd6f4ff7207b72e4a967", [:mix], [{:connection, "~> 1.0.0", [hex: :connection, optional: false]}]},
  "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.0", "edee20847c42e379bf91261db474ffbe373f8acb56e9079acb6038d4e0bf414f", [:rebar, :make], []}}

Exception:

** (DBConnection.ConnectionError) connection not available because of disconnection
    (db_connection) lib/db_connection.ex:718: DBConnection.checkout/2
    (db_connection) lib/db_connection.ex:619: DBConnection.run/3
    (db_connection) lib/db_connection.ex:463: DBConnection.prepare_execute/4
             (ecto) lib/ecto/adapters/postgres/connection.ex:78: Ecto.Adapters.Postgres.Connection.prepare_execute/5
             (ecto) lib/ecto/adapters/sql.ex:235: Ecto.Adapters.SQL.sql_call/6
             (ecto) lib/ecto/adapters/sql.ex:424: Ecto.Adapters.SQL.execute_and_cache/7
             (ecto) lib/ecto/repo/queryable.ex:127: Ecto.Repo.Queryable.execute/5
             (ecto) lib/ecto/repo/queryable.ex:40: Ecto.Repo.Queryable.all/4

Using psql and data in prod.secret.exs I'm able to connect to the database, which is located on separate server.

[fishcakez]

@nateless just to be clear you are getting this exact error:

** (Postgrex.Error) tcp connect: connection reset by peer - :econnreset

And it is occurring on OSX using docker?

[nateless]

@fishcakez, yeah, as we do logging i have this in logs, which adds ocne a second as it tries to reconnect:

....
17:12:45.838 [error] Postgrex.Protocol (#PID<0.262.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused
17:12:48.950 [error] Postgrex.Protocol (#PID<0.264.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused
17:12:50.770 [error] Postgrex.Protocol (#PID<0.269.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused
....

And no, we do not use docker, its Ubuntu 16.04.1 LTS on a dedicated server.

PS: Well after reviewing the error, we actually have a different one. Sorry for the confusion. Got a bit tired :(

[fishcakez]

@nateless do any connections succeed in connecting?

[nateless]

@fishcakez, nope. however as I've mentioned before we can connect to the database using psql on that server. I even tried to restart the app server and recompiled everything using rm -rf for deps and _build directories.

PS: exactly the same code ( but with different postgres settings in dev env ) works on my mac with local postgres.

[fishcakez]

@nateless can you access iex on the node and try something like this:

pid(0, 262, 0) |> :sys.get_state(10_000) |> get_in([:mod_state, :opts])

And verify those are the options you expect. Will need to use the relevant numbers for the pid, which would be 0 262 and 0 based on the first log entry in this post #210 (comment). The second would be 0, 264 and 0.

[nateless]

@fishcakez, after restarting postgres server, this error is gone. Idk what it was but if you need my help just ping me.

[fishcakez]

@nateless if you can reproduce and tell use what is going wrong that would be great 😉. I am unable to reproduce this.

[MartinElvar]

Having the exact same problem as @samueleaton, also trying to dockerize my app.

My setup seems to be working fine, i can connect to the postgres database fine from my elixir container, using psql. But the elixir app is just spamming, with the exact same connection information as i feed psql..

[error] Postgrex.Protocol (#PID<0.184.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused
[error] Postgrex.Protocol (#PID<0.185.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused
[error] Postgrex.Protocol (#PID<0.189.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused
[error] Postgrex.Protocol (#PID<0.187.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect: connection refused - :econnrefused

[fishcakez]

@MartinElvar is psql connecting using local socket? postgrex doesn't support that unfortunately.

OTP 19 adds experimental support for unix sockets so we likely wont add the feature until OTP 20 is released and an Elixir release is out that requires OTP 19+.

[MartinElvar]

@fishcakez My apologies, this doesn't seem to be a problem with postgrex, or for that matter Phoenix. If i run..

Postgrex.start_link(hostname: "localhost", username: "postgres", password: "postgres", database: "postgres")

I actually get a :ok, which means the problem may be else where, maybe in the Postgres Ecto adapter.

Sorry for the noise. :)

[fishcakez]

@MartinElvar FWIW Postgres.start_link/1 will return {:ok, pid} and asynchronously try to connect (and backoff to reconnect), if you add sync_connect: true, backoff_type: :stop though it will block to connect and return an error if it fails.

[MartinElvar]

@fishcakez So this means that it works then?

iex(7)> {:ok, pid} = Postgrex.start_link(hostname: "db", username: "postgres", database: "postgres", sync_connect: true, backoff_type: :stop)
{:ok, #PID<0.201.0>}

[MartinElvar]

Just for the record, here's my Ecto config.

config :users, Users.Repo,
  adapter: Ecto.Adapters.Postgres,
  database: "postgres",
  username: "postgres",
  host: "db"

[MartinElvar]

As i turns out, i copy pasted the configuration from the Ecto readme, and therefor had host over hostname, i made a PR elixir-ecto/ecto#1743. It works now 😄 .

[fishcakez]

Thanks @MartinElvar.

Just to clarify for future people, this issue is specifically about the exact log error: "connection reset by peer - :econnreset". If not this error please open a new issue.

[ericmj]

Closing this because there's currently nothing actionable for us and there haven't been any activity for two months. Please post here if you have anything to add.

[Ch4s3]

I see that this is closed, but has anyone found a good solution for this?

[pachun]

In case this will help anyone, I was seeing this error vomited to standard output on loop:

16:44:09.180 [error] Postgrex.Protocol (#PID<0.200.0>) failed to connect: ** (Postgrex.Error) FATAL 53300 (too_many_connections): too many connections for role "XXYYZZ"

when trying to heroku run iex -S mix after having deployed to heroku following these phoenix guides. I decreased my POOL_SIZE environment variable from 18 (what the guides suggest) to 10 and the next time I heroku run iex -S mix, everything worked as expected and I was dropped into a console without any errors. After I was done in the console, I switched the POOL_SIZE var back to 18.

Putting this here to help anyone else who wants a quick fix for the same problem.

I'm new to phoenix and have next to no idea what I'm doing.

[y86]

I've had similar issues trying to connect to a local postgres instance (tcp recv: timeout) on Mac OSX.

The problem for me was the following. My local postgres instance could not bind properly to the IPV4 socket (but it could bind to IPV6 socket). I was using brew to start the service and this error went quietly, I only could notice it when I manually tried to start the service.

This was the reason Phoenix/Ecto could not access the instance, but other means I had tried could (python and psql).

What was keeping postgres from binding to IPV4 socket was a process called prl_naptd which is a process from my VM software (Parallels Desktop). So by killing this process and restarting postgres service, it was able to successfully bind to both IPV4 and IPV6 sockets and then Phoenix/Ecto was able to connect.

I hope this helps other with similar problems.

[id4ho]

I always wind up here when I forget about this error.. so for myself and anyone else having issues on Heroku, the guide that @pachun mentioned offers the following solution:

heroku run "POOL_SIZE=2 iex -S mix".

My understanding is that the iex -S mix task boots your app which tries to create POOL_SIZE connections to your db. So I think the reason setting POOL_SIZE to 10 worked for @pachun above is that it gives 10 connections to your already running app and 10 to the iex task which totals 20, or the total available connections to PG on Heroku.

*The solution mentioned above assumes you set the POOL_SIZE to 18 which means 2 are leftover from the available 20 for the iex task you are running. If you set the pool size to something else you may have more or less available connections.

[joepstender]

Phoenix documentation has been updated to hexdocs.pm, you can find the link mentioned above here.