EOFError instead of SMTP error

[tobymurray-nanometrics]

Steps to reproduce

We are getting EOFError out of ActionMailer with no stack trace or useful error message. It appears to be bubbling out of deliver_now. The code below requires a real email account, so I've excluded the details here -you'll need to provide them to get the script to work. I believe the code below reproduces the symptom, but not necessarily the cause.

require 'net/smtp'

real_email_address = '' # This needs to actually be a real account
real_email_password = '' # This needs to actually be the corresponding password

#line 96 of mail-2.2.7/lib/mail/network/delivery_methods/smtp.rb
smtp = Net::SMTP.new('smtp-relay.gmail.com', 587)
smtp.enable_starttls_auto if smtp.respond_to?(:enable_starttls_auto)
smtp.set_debug_output $stderr

smtp.start('does-not-matter', real_email_address, real_email_password, 'Login') do |smtp|
  smtp.sendmail('Message does not matter', 'inconsistent from address', 'does not matter')
end

To be clear, it seems the error that arises when the from address is not appropriately authorized when coupled with the email address used to create the SMTP session. In the above code that discrepancy is captured by the difference between real_email_address and inconsistent from address (which is presumably not authorized to send from real_email_address).

Expected behavior

Some form of actionable error, likely Net::SMTPError of some sort.

Actual behavior

Connection opened: smtp-relay.gmail.com:587
-> "220 smtp-relay.gmail.com ESMTP o192sm21049545itb.4 - gsmtp\r\n"
<- "EHLO does-not-matter\r\n"
-> "250-smtp-relay.gmail.com at your service, [***.***.***.***]\r\n"

<snip interaction>

TLS connection started
<- "EHLO does-not-matter\r\n"
-> "250-smtp-relay.gmail.com at your service, [***.***.***.***]\r\n"

<snip interaction>

-> "235 2.7.0 Accepted\r\n"
<- "MAIL FROM:<inconsistent from address>\r\n"
-> "550-5.7.1 Invalid credentials for relay [***.***.***.***]. The IP address you've\r\n"
-> "550-5.7.1 registered in your G Suite SMTP Relay service doesn't match domain of\r\n"
-> "550-5.7.1 the account this email is being sent from. If you are trying to relay\r\n"
-> "550-5.7.1 mail from a domain that isn't registered under your G Suite account\r\n"
-> "550-5.7.1 or has empty envelope-from, you must configure your mail server\r\n"
-> "550-5.7.1 either to use SMTP AUTH to identify the sending domain or to present\r\n"
-> "550-5.7.1 one of your domain names in the HELO or EHLO command. For more\r\n"
-> "550-5.7.1 information, please visit\r\n"
-> "550 5.7.1  https://support.google.com/a/answer/6140680#invalidcred o192sm21049545itb.4 - gsmtp\r\n"
<- "QUIT\r\n"
EOFError: end of file reached

System configuration

Rails version:
Rails 4.2.7.1

Ruby version:
jruby 9.1.6.0 (2.3.1) 2016-11-09 0150a76 OpenJDK 64-Bit Server VM 25.111-b14 on 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14 +jit [linux-x86_64]

[rafaelfranca]

Can you please provide a sample application that reproduces the error?

[tobymurray-nanometrics]

Same caveat here, user_name and password need to be filled in with valid credentials.

require 'action_mailer'

ActionMailer::Base.smtp_settings = {
    :address              => "smtp-relay.gmail.com",
    :port                 => 587,
    :domain               => "gmail.com",
    :user_name            => "", # Real email account
    :password             => "", # Password that corresponds to that email account
    :authentication       => "Login",
    :enable_starttls_auto => true
}

class TestMailer < ActionMailer::Base
  def mail(args)
    super
  end
end

TestMailer.mail(from: 'inconsistent from address', to: 'real+email+address@gmail.com', subject: 'does not matter', body: 'does not matter').deliver_now

yields

$ ruby test.rb 
EOFError: end of file reached

[tobymurray-nanometrics]

To be clear, @rafaelfranca, my understanding is both the original script I provided and the second one are the same issue. The second example is closer to how we first encountered it - a very mysterious looking error out of an ActionMailer subclass. The first example is our attempt at a minimal reproduction. I'm not aware of an executable script that can be posted without passing in the username and password parameters in some respect - I hope it's easy enough to understand as written.

[matthewd]

If you can reproduce the problem directly with net/smtp, then it seems to follow that it's a bug in that library... no?

[jeremy]

Looks like Google is dropping the connection unceremoniously. The SMTP client sends a QUIT and waits for a response which will never come.

EOFError is the expected exception for IO reads that expect a result but hit the end of the stream instead, and Net::SMTP#start documents that it may raise IOError (EOFError superclass).

Whether bubbling up the underlying net/protocol exceptions useful for developers is… a good question! Action Mailer uses Mail which uses Net::SMTP which uses Net::Protocol. Arguably, this EOFError could be rescued and raised as a protocol error from Net::SMTP then rescued and raised as a Mail delivery error rather than exposing the underlying transport error.

[tobymurray-nanometrics]

If you can reproduce the problem directly with net/smtp, then it seems to follow that it's a bug in that library... no?

Agreed-ish. Whether net/smtp is erroneously letting the EOF escape is certainly something.

Arguably, this EOFError could be rescued and raised as a protocol error from Net::SMTP then rescued and raised as a Mail delivery error rather than exposing the underlying transport error.

This is the impetus for the ticket - it took a non-trivial amount of time to find out where the error was coming from (recall there's no stack trace attached), and so it came across as a deficiency in Rails from my perspective.

I can move this to Ruby or duplicate there as appropriate.

[matthewd]

Even though Google is violating the RFC ("The receiver MUST NOT intentionally close the transmission channel until it receives and replies to a QUIT command (even if there was an error).") by dropping the connection as soon as they send the 550, IMO net/smtp should probably ignore the protocol violation in favour of passing along the legitimate error it's already received.

An EOFError in general (at some other, not-trying-to-quit-anyway, time) doesn't strike me as entirely unreasonable to expose... but the missing backtrace seems improvable.

At that point, I don't think we would need to do anything further: the Net::SMTPError in this case, and EOFError in the event of a more outright protocol violation, feels pretty consistent with how we handle exceptional circumstances elsewhere.

[tobymurray-nanometrics]

I opened a similar Ruby ticket to deal with the net/smtp letting the EOF out: https://bugs.ruby-lang.org/issues/13018

If that's addressed on their end, I imagine the ActionMailer code would report it properly - if not, catching it and rethrowing the error with a stack trace would be a nice improvement in my view.

[pixeltrix]

FYI, there's a whole bunch of non-net/smtp errors that leak out - here's a list of exceptions that we've encountered on petition.parliament.uk:

  PERMANENT_FAILURES = [
    Net::SMTPFatalError,
    Net::SMTPSyntaxError
  ]

  TEMPORARY_FAILURES = [
    Net::SMTPAuthenticationError,
    Net::OpenTimeout,
    Net::SMTPServerBusy,
    Errno::ECONNRESET,
    Errno::ECONNREFUSED,
    Errno::ETIMEDOUT,
    Timeout::Error,
    EOFError
  ]

There's a couple more to add to that list - OpenSSL::SSL::SSLError and SocketError. These occur when the DNS network is under a DDoS attack 😭

[tobymurray-nanometrics]

Someone left a comment on the Ruby bug saying it's unlikely a change (if any is to be made) would make it into Ruby 2.4. Perhaps worth taking a look at handling in the mean time to keep the exception from escaping ActionMailer?

[pixeltrix]

One thing we may be able to do in Action Mailer is to improve the behaviour of the job class that handles deliver_later. Currently it's pretty basic and all of the above errors leak out and generally in apps I write custom jobs to handle email delivery.

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 5-1-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.

[sachin-metacube]

I am facing the same issue,

I have my own domain and i have got gsuite acoount, i created an email account on that and used it too send emails, and getting this error. Error: end of file reached

My SMTP settings are:

ActionMailer::Base.smtp_settings = {
  address: 'smtp.gmail.com',
  port: '465',
  domain: 'gmail.com',
  user_name: notifications@mydomain.com,
  password: 'mypassword',
  authentication: 'plain',
  enable_starttls_auto: true
}

[ianrandmckenzie]

I am getting this silent failure as well. All I get is the EOFError. Like sachin-metacube, I am using my own domain through a gsuite account. I wonder if this is a problem with Google that has started fairly recently? I was using identical settings May 13th (15 days ago), and they have stopped working.

Is this an issue I should be bringing up with Google support?

[ianrandmckenzie]

@sachin-metacube

I fixed the problem I was having. Maybe this will help you.

Step 1. Follow this guide here:
https://support.google.com/a/answer/2956491
-> Be sure to enable TLS
-> Make sure you're using port 587 with TLS

Here are my Devise mailer settings:

  config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }
  config.action_mailer.delivery_method = :smtp
  config.action_mailer.perform_deliveries = true
  config.action_mailer.raise_delivery_errors = true
  config.action_mailer.smtp_settings =
  {

    :address            => 'smtp-relay.gmail.com',
    :port               => 587,
    :domain             => 'gmail.com',
    :authentication     => :plain,
    :user_name          => ENV['MAILER_ADDRESS'],
    :password           => ENV['MAILER_SECRET'],
    :enable_starttls_auto => true
  }

[overdrivemachines]

Yes changing the port from 465 to 587 worked for me. Here is the real guide for gmail users if you are sending from @gmail.com:
https://support.google.com/mail/answer/7104828?hl=en&visit_id=1-636673592167320912-3397805049&rd=1

[edgartheunready]

I had a EOFError as well. I was using the wrong port number (was using the smtp port when I was trying to use imap). Hopefully that helps the next person :)

[SuprajaSuresh]

I faced the same issue with Gmail SMTP. Adding

domain: 'mydomain.com'
to the SMTP configuration fixed it for me.

[x2es]

In my case the issue was is sitting in middle proxy which blocks smtp.
Some tips about diagnose: https://stackoverflow.com/a/64560726/983232

[ianrandmckenzie]

@SuprajaSuresh I came back to this issue over 4 years later after running into the same problem again working on a new app. Figured my past self could help. It did! And so did your addition. Thank you 🙏

[SuprajaSuresh]

@SuprajaSuresh I came back to this issue over 4 years later after running into the same problem again working on a new app. Figured my past self could help. It did! And so did your addition. Thank you pray

Glad to know that it helped !!!