gen_server_client:send send_error, closed

[jschoch]

I recently upgraded from 0.15 to 1.1.1 and i'm getting the following behavior for some emails.

18:49:08.532 [warn]  DateTime

18:49:08.532 [warn]  "2021-11-20 18:49:08.532615Z"

18:49:08.534 [warn]  handle error:

18:49:08.534 [warn]  %{
  class: :send_error,
  details: :closed,
  state: %MailToJson.SmtpHandler.State{options: [parse: true]}
}

18:49:08.534 [warn]  terminated

18:49:08.535 [warn]  %{
  reason: {:send_error, :closed},
  state: %MailToJson.SmtpHandler.State{options: [parse: true]}
}

18:49:08.535 [error] GenServer #PID<0.377.0> terminating
** (stop) {:send_error, :closed}
Last message: {:tcp, #Port<0.1147>, "QUIT\r\n"}

18:49:08.535 [error] Process #PID<0.377.0> terminating
** (exit) {:send_error, :closed}
    (stdlib) gen_server.erl:751: :gen_server.handle_common_reply/8
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Initial Call: :gen_smtp_server_session.ranch_init/1
Ancestors: [#PID<0.283.0>, #PID<0.278.0>, #PID<0.277.0>, :ranch_sup, #PID<0.253.0>]

18:49:08.536 [error] Ranch listener :gen_smtp_server had connection process started with :gen_smtp_server_session:start_link/3 at #PID<0.377.0> exit with reason: {:send_error, :closed}


18:49:08.734 [warn]  callback:

18:49:08.734 [warn]  {:ok, "2.0.0 Ok: queued as 8759240722\r\n"}

i was only using send/2 before and the client was not actually sending anything. I added an implementation of handle_error in the server and added a callback to gen_smtp_client.send/3 and it will actually send but it still throws this error. Note that callback seems to have an ok result.

[seriyps]

I really have difficulties understanding what are you trying to do. Are you sending some email with gen_smtp_client:send to the SMTP server running with gen_smtp_server?
If so, could you at least tell us what options do you use for send and to start the server?
If you have some code to share?

[jschoch]

It sets up a smtp server. Postfix will get mails and send them depending on the domain. The server parses the messages and rewrites the messages to route them base on a rulset. When all the processing is complete it calls gen_smtp_client.send/3 to either send a bounce or send a new rewritten message back through the postfix server. This worked fine for several years prior to the upgrade to 1.1.1.

Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:1] [hipe]
Elixir (1.9.1)

defmodule MailToJson.SmtpServer do

  def start_link do
    session_options = [ callbackoptions: [parse: true] ]
    smtp_port = IdgonMail.config(:smtp_port)
    server_config =
       [
         port: smtp_port,
         protocol: :tcp,
         domain: "example.com",
         auth: :never,
         tls: :never,
         address: {0, 0, 0, 0}
       ]
   
    :gen_smtp_server.start(MailToJson.SmtpHandler, [server_config: server_config, sessionoptions: session_options])
  end

end

the smtp handler

def handle_DATA(from, to, data, state) do
    unique_id = Base.encode16(:erlang.md5(data), case: :lower)
    
    try do
      case Idg.Email.process(data,orig_url) do
        {:bounce, to, mail_pipe} ->
          I.le "email bounce from: ", mail_pipe.from
          {:error, "550 5.1.1 : Recipient address rejected: User unknown", state}
        stuff ->
          Idg.Email.send(stuff)
          {:ok, unique_id, state}
      end
    rescue
      e in IdgonNotFound ->
        I.le "Idgon was not found ", {e,state}
        {:error, "550 5.1.1 : Recipient address rejected: User unknown", state}
      e in RuntimeError ->
        I.le "something blew up ", {e,state}
        reraise e, __STACKTRACE__
        {:error, "111 problems on my end", state}
      e ->
        I.le "something blew up ", {e,state}
        reraise e, __STACKTRACE__
          {:error, "111 problems on my end", state}
    end
  end

Idg.Email.send

  def send({:error,_,_}) do
    :error
  end

  def send(:bounce, to, mail_pipe) do
    port = 25
    host = "localhost"
    email = """
            From: Postmaster <bounce@idgon.com>
            To: #{mail_pipe.from.old.name} #{mail_pipe.from.old.address}
            Subject: Bounce

            email bounced
            """
    client_options = [relay: host, username: "ec2-user", password: "anything", port: port]
    :gen_smtp_client.send(email, client_options)
  end

  def send(email) do
    port = 25
    host = "localhost"
    client_options = [relay: host, username: "mailer", password: "anything", port: port,retries: 2]
    
    #  I.lw is my logger function
    callback = fn(r) -> I.lw( "callback: ", r) end
    :gen_smtp_client.send(email, client_options,callback)
  end

[taobojlen]

+1; I'm getting this same error for pretty much every email received with my gen_smtp SMTP server. I'm only receiving, not sending. It doesn't seem to actually break anything since I'm receiving the email data correctly. Here's my DATA handler:

  def handle_DATA(from, to, data, state) do
    Logger.info("Received email from #{from} with data #{data}")

    %{from: from, to: hd(to), data: data}
    |> EmailHandler.new()
    |> Oban.insert()

    {:ok, "1", state}
  end

and my config options:

config :shroud, :mailer,
  smtp_options: [
    port: 1587,
    sessionoptions: [
      hostname: "app.shroud.email",
      certfile:
        "path/to/crt",
      keyfile:
        "path/to/key"
    ],
    tls_options: [
      # Don't verify peers (we'll forward anything we can)
      verify: :verify_none,
      log_level: :info
    ]
  ]

If it's helpful, the full code is available here: https://gitlab.com/shroud/shroud.email/-/blob/main/lib/shroud/email/smtp_server.ex

[jschoch]

anyone have any ideas on this one? I'm still seeing sporadic issues. do you need a more fully flushed out example to test against?

[seriyps]

@jschoch I see you are using gen_smtp_client.send(email, client_options) and this API is non-blocking, but it also links to the calling process.
It means that if SMTP server session process would exit with some error before the process started by send done sending email, this sender process will be killed as well. That might be one of the reasons. I would suggest to try to use send_blocking instead.

Another thing you may try to do is to suppress the send_error type of errors. See #270 (comment) and #270 (comment) for details.