New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NameError: uninitialized constant Mail::Parsers::ContentTypeParser #912

Closed
hiattp opened this Issue Sep 27, 2015 · 45 comments

Comments

Projects
None yet
@hiattp
Copy link

hiattp commented Sep 27, 2015

Just saw this error in our Rails 4.2.2 app running mail 2.6.3 on ruby 2.0.0. Any suggestions on how to track it down? Thanks!

@claudiofullscreen

This comment has been minimized.

Copy link

claudiofullscreen commented Sep 28, 2015

Me too! Here's the backtrace from Honeybadger. I hope it helps:

NameError: uninitialized constant Mail::Parsers::ContentTypeParser

Full Backtrace

content_type_element.rb  8  initialize(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/elements/content_type_element.rb:8:in `initialize'
content_type_field.rb  49  new(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:49:in `new'
content_type_field.rb  49  rescue in attempt_to_clean(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:49:in `rescue in attempt_to_clean'
content_type_field.rb  46  attempt_to_clean(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:46:in `attempt_to_clean'
content_type_field.rb  40  rescue in element(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:40:in `rescue in element'
content_type_field.rb  37  element(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:37:in `element'
content_type_field.rb  32  parse(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:32:in `parse'
content_type_field.rb  24  initialize(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:24:in `initialize'
field.rb  239  new(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/field.rb:239:in `new'
field.rb  239  new_field(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/field.rb:239:in `new_field'
field.rb  228  create_field(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/field.rb:228:in `create_field'
field.rb  141  field(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/field.rb:141:in `field'
field.rb  189  method_missing(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/field.rb:189:in `method_missing'
message.rb  1196  default(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/message.rb:1196:in `default'
message.rb  594  content_type(...)
[GEM_ROOT]/gems/mail-2.6.3/lib/mail/message.rb:594:in `content_type'
base.rb  871  set_content_type(...)
[GEM_ROOT]/gems/actionmailer-4.2.4/lib/action_mailer/base.rb:871:in `set_content_type'
base.rb  835  mail(...)
@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Sep 30, 2015

Could you do some troubleshooting in console or a debugger to diagnose what's happening?

@claudiofullscreen

This comment has been minimized.

Copy link

claudiofullscreen commented Sep 30, 2015

Unfortunately I was not able to replicate the error.
Only happened that day three times… and then the same code worked as before.

The only hint that I have is that the title of the email includes an emoji.
It still does, and the same code now works… I just wonder if that was the case for @hiattp too.

I'll update this ticket in case I'm able to replicate the error in the future.

@hiattp

This comment has been minimized.

Copy link

hiattp commented Sep 30, 2015

We don't have emojis. Unfortunately this will be a hard one to track down in the console. It seems related to rails autoloading and is sporadic, so probably depends on the order in which things run on a new process. Will try to pin it down if we see another case

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Sep 30, 2015

Thanks!

@scottwater

This comment has been minimized.

Copy link

scottwater commented Sep 30, 2015

I have seen this off and on over the last couple of weeks (Rails 4.2.1), but I have not been able to track it down. First time it popped up was when we made a switch to Ruby 2.2.3.

@BrentWheeldon

This comment has been minimized.

Copy link

BrentWheeldon commented Oct 7, 2015

We are also sporadically seeing this, but for Mail::Parsers::AddressStruct. We also recently updated to Ruby 2.2.3.

@stephankaag

This comment has been minimized.

Copy link

stephankaag commented Oct 8, 2015

Same issue overhere

-> NameError: uninitialized constant Mail::Parsers::ContentTypeParser
mail-2.6.3/lib/mail/elements/content_type_element.rb:8→ initialize
mail-2.6.3/lib/mail/fields/content_type_field.rb:49→ new
mail-2.6.3/lib/mail/fields/content_type_field.rb:49→ rescue in attempt_to_clean
mail-2.6.3/lib/mail/fields/content_type_field.rb:46→ attempt_to_clean
mail-2.6.3/lib/mail/fields/content_type_field.rb:40→ rescue in element
mail-2.6.3/lib/mail/fields/content_type_field.rb:37→ element
mail-2.6.3/lib/mail/fields/content_type_field.rb:32→ parse
mail-2.6.3/lib/mail/fields/content_type_field.rb:24→ initialize
mail-2.6.3/lib/mail/field.rb:239→ new
mail-2.6.3/lib/mail/field.rb:239→ new_field
mail-2.6.3/lib/mail/field.rb:228→ create_field
mail-2.6.3/lib/mail/field.rb:167→ update
mail-2.6.3/lib/mail/header.rb:174→ []=
mail-2.6.3/lib/mail/message.rb:598→ content_type=    
@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Oct 13, 2015

I got the same error too, but backtrace is different:
The most strange thing is that it only happens sometimes, can't be sure how to reproduce it

mail-2.6.3/lib/mail/elements/content_type_element.rb:8→ initialize
mail-2.6.3/lib/mail/fields/content_type_field.rb:49→ new
mail-2.6.3/lib/mail/fields/content_type_field.rb:49→ rescue in attempt_to_clean
mail-2.6.3/lib/mail/fields/content_type_field.rb:46→ attempt_to_clean
mail-2.6.3/lib/mail/fields/content_type_field.rb:40→ rescue in element
mail-2.6.3/lib/mail/fields/content_type_field.rb:37→ element
mail-2.6.3/lib/mail/fields/content_type_field.rb:32→ parse
mail-2.6.3/lib/mail/fields/content_type_field.rb:24→ initialize
mail-2.6.3/lib/mail/field.rb:239→ new
mail-2.6.3/lib/mail/field.rb:239→ new_field
mail-2.6.3/lib/mail/field.rb:228→ create_field
mail-2.6.3/lib/mail/field.rb:141→ field
mail-2.6.3/lib/mail/field.rb:189→ method_missing
mail-2.6.3/lib/mail/message.rb:1196→ default
mail-2.6.3/lib/mail/message.rb:594→ content_type
actionmailer-4.1.13/lib/action_mailer/base.rb:831→ set_content_type
actionmailer-4.1.13/lib/action_mailer/base.rb:795→ mail
mandriller-0.2.0/lib/mandriller/base.rb:80→ mail
@airimie

This comment has been minimized.

Copy link

airimie commented Nov 9, 2015

We got this error a few times now. We are using sidekiq to send emails and sendgrid to deliver.

NameError: uninitialized constant Mail::Parsers::ContentTypeParser

/gems/mail-2.6.3/lib/mail/elements/content_type_element.rb:8 in initialize
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:49 in new
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:49 in rescue in attempt_to_clean
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:46 in attempt_to_clean
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:40 in rescue in element
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:37 in element
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:32 in parse
/gems/mail-2.6.3/lib/mail/fields/content_type_field.rb:24 in initialize
/gems/mail-2.6.3/lib/mail/field.rb:239 in new
/gems/mail-2.6.3/lib/mail/field.rb:239 in new_field
/gems/mail-2.6.3/lib/mail/field.rb:228 in create_field
/gems/mail-2.6.3/lib/mail/field.rb:167 in update
/gems/mail-2.6.3/lib/mail/header.rb:174 in []=
/gems/mail-2.6.3/lib/mail/message.rb:598 in content_type=
/gems/actionmailer-4.2.4/lib/action_mailer/base.rb:835 in mail
/gems/sendgrid-1.2.0/lib/sendgrid.rb:168 in mail
app/mailers/training_mailer.rb:7 in training_notification

@jamilbk

This comment has been minimized.

Copy link

jamilbk commented Nov 9, 2015

Getting the same error here. Difficult to reproduce.

@airimie

This comment has been minimized.

Copy link

airimie commented Nov 9, 2015

require 'mail/elements' is called after register_autoload :Parsers, "mail/parsers" (which probably it is not thread safe - looking at http://rxr.whitequark.org/mri/source/variable.c#1597 the only thing that method is doing is to add to a table the files to be loaded. This looks very much like the rails autoloading hell :) http://urbanautomaton.com/blog/2013/08/27/rails-autoloading-hell/ The solution will be to not use the autoloading at all.

@nburt

This comment has been minimized.

Copy link

nburt commented Nov 9, 2015

Haven't been able to reproduce this either but getting this occasionally with various different mailers.

Backtrace:

mail-2.6.3/lib/mail/elements/content_type_element.rb:8→ initialize
mail-2.6.3/lib/mail/fields/content_type_field.rb:49→ new
mail-2.6.3/lib/mail/fields/content_type_field.rb:49→ rescue in attempt_to_clean
mail-2.6.3/lib/mail/fields/content_type_field.rb:46→ attempt_to_clean
mail-2.6.3/lib/mail/fields/content_type_field.rb:40→ rescue in element
mail-2.6.3/lib/mail/fields/content_type_field.rb:37→ element
mail-2.6.3/lib/mail/fields/content_type_field.rb:32→ parse
mail-2.6.3/lib/mail/fields/content_type_field.rb:24→ initialize
mail-2.6.3/lib/mail/field.rb:239→ new
mail-2.6.3/lib/mail/field.rb:239→ new_field
mail-2.6.3/lib/mail/field.rb:228→ create_field
mail-2.6.3/lib/mail/field.rb:141→ field
mail-2.6.3/lib/mail/field.rb:189→ method_missing
mail-2.6.3/lib/mail/message.rb:1196→ default
mail-2.6.3/lib/mail/message.rb:594→ content_type
actionmailer-4.2.4/lib/action_mailer/base.rb:871→ set_content_type
actionmailer-4.2.4/lib/action_mailer/base.rb:835→ mail
roadie-rails-1.0.6/lib/roadie/rails/automatic.rb:5→ mail
app/mailers/testing_mailer.rb:10→ test_passed

@mirjoy

This comment has been minimized.

Copy link

mirjoy commented Nov 12, 2015

I was able to resolve this issue by setting the content-type in my email:
content_type: "text/html"

@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Nov 27, 2015

I just add this workaround in an initializer:

require "mail/parsers/content_type_parser"

Already tested in staging for one day and no error pops up anymore
Now I will put this on production and see if the error happens again for the following few days

@wallace

This comment has been minimized.

Copy link

wallace commented Nov 27, 2015

I'm also experiencing this issue: I have the following versions:

ruby 2.2.3
rails 4.2.4
mail 2.6.3
sidekiq 3.2.6

@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Nov 30, 2015

It seems the project has not encountered this issue anymore since the workaround is applied.

@mockdeep

This comment has been minimized.

Copy link

mockdeep commented Nov 30, 2015

We just started running into this issue. We haven't upgraded rails in a while, so not sure why it started happening now. Maybe we're using mailers differently than we were before.

@QuotableWater7

This comment has been minimized.

Copy link

QuotableWater7 commented Dec 1, 2015

@PikachuEXE do you explicitly load anything else before the content type parser? I tried your solution and see this during bootup:

mail-2.6.3/lib/mail/parsers/content_type_parser.rb:1:in `<top (required)>':  uninitialized constant Mail (NameError)
@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Dec 2, 2015

I guess I should post the whole thing :P
The full content is

require "mail"
# This is trying to workaround an issue in `mail` gem
# Issue: https://github.com/mikel/mail/issues/912
#
# Since with current version (2.6.3) it is using `autoload`
# And as mentioned by a comment in the issue above
# It might not be thread-safe and
# might have problem in threaded environment like Sidekiq workers
#
# So we try to require the file manually here to avoid
# "uninitialized constant" error
#
# This is merely a workaround since
# it should fixed by not using the `autoload`
require "mail/parsers/content_type_parser"
@QuotableWater7

This comment has been minimized.

Copy link

QuotableWater7 commented Dec 2, 2015

ha thanks! I almost tried that but got sucked into something else and went the lazy route

@ttasanen

This comment has been minimized.

Copy link

ttasanen commented Dec 18, 2015

We just had this issue also. It seem to happen randomly in multi threaded environment. Manually requiring the file solved the issue.

register_autoload definitely is not thread safe.

@schneems

This comment has been minimized.

Copy link

schneems commented Jan 8, 2016

Seeing this with mail 2.6.3 and Ruby 2.3.0.

2016-01-08T20:48:05.760321+00:00 app[web.1]: NameError (uninitialized constant Mail::Parsers::AddressStruct):
2016-01-08T20:48:05.760321+00:00 app[web.1]:   app/models/user.rb:102:in `valid_email?'
2016-01-08T20:48:05.760322+00:00 app[web.1]:   app/controllers/users/omniauth_callbacks_controller.rb:6:in `github'

This constant is being expected in the validate_email gem.

@schneems

This comment has been minimized.

Copy link

schneems commented Jan 8, 2016

Was able to force a fix in the Gemfile codetriage/codetriage@4186732. I tried to get this to reproduce locally, i used forks, threads, you name it. Still not sure what the failure conditions are for this. Even stranger is that this code was working fine yesterday and I didn't deploy anything new. Strange bugs are strange.

@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Apr 26, 2016

I found an old PR #406 what actually added an API to load all components
I am going to use that instead of the code I posted earlier

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Apr 26, 2016

Mail.eager_autoload!

😁

@jeremy jeremy closed this Apr 26, 2016

@schneems

This comment has been minimized.

Copy link

schneems commented Apr 29, 2016

I still think there is a bug here with the auto loading code. People shouldn't have to know about this custom API. I'm guessing that command loads EVERYTHING. Which can take up quite a bit of ram if you don't need the mail parsers 😓

@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Apr 29, 2016

Yup but I guess this is an autoload bug which is not this gem's problem?
Not sure :P

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Apr 29, 2016

Right. We can play some whack-a-mole to address specific load concurrency issues if they're more common than most, but we can't address them all, so we offer an API to load the full library as if it had been Kernel#require. Making that more fine-grained (like separate eager_load for parsing and generating) is a good idea for those who'd know to take advantage of it.

@schneems

This comment has been minimized.

Copy link

schneems commented Apr 29, 2016

I use lots of gems with autoload and this is the only project that ever exhibited this behavior. I think that something in this project is interacting with autoload somehow.

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Apr 30, 2016

I think you're right, @schneems.

@ajahongir

This comment has been minimized.

Copy link

ajahongir commented May 6, 2016

having same issue on production, using:
rails 4.2.1
mail 2.6.4
sidekiq: 4.1.1

@schneems

This comment has been minimized.

Copy link

schneems commented May 10, 2016

Crazy idea here. Can we split up the two gems to mail and mail-parsers. The lazy loading of parsers works okay in keeping memory down for people who aren't parsing and don't need them to be loaded. However people who are parsing and don't know that the code is lazy loaded will not see CoW benefits while forking if the code is loaded at runtime (after boot time).

Would be nice to make the behavior of when a parser is required or not more obvious through adding a gem to the gemfile.

@mvdamme

This comment has been minimized.

Copy link

mvdamme commented Jun 20, 2016

I'm seeing the same issue (with mail 2.6.3, rails 4.2.5.1 and ruby 2.2.4). It happens when sending many emails concurrently in different threads using sucker_punch.
No problems when not sending concurrently.

@hakunin

This comment has been minimized.

Copy link

hakunin commented Dec 1, 2016

Just saw this, we use sidekiq and ruby 2.2.5

@mdesjardins

This comment has been minimized.

Copy link

mdesjardins commented Dec 2, 2016

I apologize for the "me too" comment but looking through the comments I think I might be the first person to report this problem using Rails 5, so perhaps this is a tiny bit helpful (?):

rails 5.0.0.1
sidekiq 4.2.5
ruby 2.2.3
mail 2.6.4

@schneems

This comment has been minimized.

Copy link

schneems commented Dec 2, 2016

I'm still thinking the best way forward is to split out into two separate gems.

@skatenerd

This comment has been minimized.

Copy link

skatenerd commented Dec 6, 2016

@jeremy I haven't used rails in a while. Did you put the eager_autoload! call in a Rails Initializer?

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Dec 6, 2016

@skatenerd Yep!

Anyone tried with the master branch? It uses an explicit require now: 0778261#diff-28c5c174388b9b89993a71d813ea1e0bR3

@schneems

This comment has been minimized.

Copy link

schneems commented Dec 6, 2016

Doesn't that kill our savings we got from #815 ?

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Dec 6, 2016

@schneems ContentTypeElement is still autoloaded, but its dependency (ContentTypeParser) is not. So this kicks the can a bit, not eliminating concurrent autoload collision but possibly reducing its frequency.

Working to get parser mem usage down. That's a bit more of a challenge with Ragel's quicker generated code taking up a lot of … code. Generating C/Java and binding to it would be far slimmer.

@schneems

This comment has been minimized.

Copy link

schneems commented Dec 6, 2016

Splitting this lib out into two gems would kill the race condition and help with the CoW by loading the parsers earlier for those who need them.

@jeremy

This comment has been minimized.

Copy link
Collaborator

jeremy commented Dec 6, 2016

@schneems I hear you :) Splitting into entirely separate gems is a few steps further than needed, though.

More likely path is to dial back on autoloading (e.g. for Parsers) and expect explicit requires from callers. Thanks a major breaking API change.

Also, note that parser usage isn't limited to parsing incoming email. It's also used to build outgoing emails (since the parsers are for field values) so, for most users, splitting out parsers may not yield the savings anticipated.

@johvet

This comment has been minimized.

Copy link

johvet commented Dec 3, 2017

Have you guys considered adding Mail to config.eager_load_namespaces. See http://edgeguides.rubyonrails.org/configuring.html

@PikachuEXE

This comment has been minimized.

Copy link

PikachuEXE commented Dec 3, 2017

I rather just call custom eager load method in an initializer
config.eager_load_namespaces limits the API to eager_load! with no arguments at all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment