Possible memory leak in prepend_view_path #14301

Closed
r38y opened this Issue Mar 6, 2014 · 14 comments

Projects

None yet

7 participants

@r38y
r38y commented Mar 6, 2014

I think I'm seeing a memory leak when using prepend_view_path.

When I use the instance method in a controller:

def setup_view_paths
  prepend_view_path(magazine_views)
end

I get a lot of leaked strings (and other objects in the same area)

Leaked 1384 STRING objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:299
Leaked 692 DATA objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:299
Leaked 616 ARRAY objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:299
Leaked 396 ARRAY objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:300
Leaked 395 DATA objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:106
Leaked 235 STRING objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:332
Leaked 197 NODE objects at: /Users/r38y/.rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/actionpack-4.0.2/lib/action_view/template.rb:299

That look like _app_views_site_overrides_j____com_posts_shared__featured_html_erb___3233500999093705389_70234272758740.

When I use the class method in an initializer:

ActionController::Base.prepend_view_path('app/views/site_overrides/j-14.com')

I don't leak anything in template.rb.

I need to use the instance method because I need to set things up for themes.

I've seen this in production and development.

This is how I'm getting that information: https://gist.github.com/r38y/02f4378a19f9064d2da2

I've been in the Rails source code for a couple days and cannot figure out why it is leaking 😦. Any ideas would would a ton. Thank you!

@r38y
r38y commented Mar 6, 2014

If I render the partial like:

<%= render 'site_overrides/j-14.com/posts/shared/featured', post: post %>

I don't see the issue.

@robin850 robin850 added the actionview label Mar 6, 2014
@r38y
r38y commented Mar 6, 2014

It looks like it is the string for the method name that is leaking

If I change

# Make sure that the resulting String to be evalled is in the
# encoding of the code
source = <<-end_src
  def #{method_name}(local_assigns, output_buffer)
    _old_virtual_path, @virtual_path = @virtual_path, #{@virtual_path.inspect};_old_output_buffer = @output_buffer;#{locals_code};#{code}
  ensure
    @virtual_path, @output_buffer = _old_virtual_path, _old_output_buffer
  end
end_src

to

source = ''
source << "def #{method_name}(local_assigns, output_buffer)\n"
source << "_old_virtual_path, @virtual_path = @virtual_path, #{@virtual_path.inspect};_old_output_buffer = @output_buffer;#{locals_code};#{code}\n"
source << "ensure\n"
source << "@virtual_path, @output_buffer = _old_virtual_path, _old_output_buffer\n"
source << 'end\n'

I no longer the partials memory leaks. I seem to only get the main template method name leak:

_app_views_site_overrides_j____com_pages_index_html_erb___116067111201229435_70221803182180
@r38y
r38y commented Mar 6, 2014

Changing

source = <<-end_src
  def #{method_name}(local_assigns, output_buffer)
    _old_virtual_path, @virtual_path = @virtual_path, #{@virtual_path.inspect};_old_output_buffer = @output_buffer;#{locals_code};#{code}
  ensure
    @virtual_path, @output_buffer = _old_virtual_path, _old_output_buffer
  end
end_src

to

source.freeze = <<-end_src
  def #{method_name}(local_assigns, output_buffer)
    _old_virtual_path, @virtual_path = @virtual_path, #{@virtual_path.inspect};_old_output_buffer = @output_buffer;#{locals_code};#{code}
  ensure
    @virtual_path, @output_buffer = _old_virtual_path, _old_output_buffer
  end
end_src

Seems to fix my template leak problem

@r38y
r38y commented Mar 6, 2014

Turns out, #freeze didn't work. I didn't have a sanity check that the page was actually rendering.

@r38y
r38y commented Mar 6, 2014

Even thought he #freeze was a wrong fix, there is still a difference if I use the class method vs the instance method

I dumped the view_paths after each of three requests using both techniques to see if there were any differences:

Using the class method in an initializer vs instance method in a before filter: https://gist.github.com/r38y/9400923

The difference I noticed is using the class method resulted in exactly the same instances for each of the objects but the instance method version resulted in different instance methods for

ActionView::OptimizedFileSystemResolver
ActionView::Resolver::Cache
ActionView::Resolver::Cache::SmallCache

I think it makes sense (maybe) but I don't think the strings associated with them are being released.

@JustMikey

I've just noticed the same issue in my app with prepend_view_path and you saved me a lot of time by saying that it can be put it in an initializer, I didnt even think of it somehow :) Hope they'll fix it

@r38y
r38y commented Mar 15, 2014

I'm glad it helped a bit!

I have quite a few overrides depending on the request so the way I got around it was to create a registry in the initializer:

PATH_SET_REGISTRY = {}
Dir.entries('app/views/site_overrides').reject{|e| e.in?(['.', '..'])}.each do|domain|
  paths = ["app/views/site_overrides/#{domain}"]
  PATH_SET_REGISTRY[domain] = ActionView::PathSet.new(paths)
end
PATH_SET_REGISTRY

And then look up the override paths in that registry on each request:

def setup_view_paths
  prepend_view_path PATH_SET_REGISTRY[current_magazine.domain]
end

I still think there is a memory leak in Rails because I think we should be using the same path and cache instances for each lookup path between requests.

@rafaelfranca
Member
@tenderlove
Member

Can you prove the objects are actually being leaked by putting this code in an infinite loop and showing unbounded memory growth?

Also a sample app would really help.

@r38y
r38y commented Mar 28, 2014

I created a fresh, small app that has one action with prepend_view_path and one that doesn't:

https://github.com/r38y/prepend_view_path_example

/prepended has prepend_view_path and /not_prepended does not.

I used apache bench to hit it a bunch of times and it seemed like the one grew more than the other one, BUT, it is hard to see when the templates are so small. Our app is ~huge.

Maybe someone can help me figure out how to get more info from it?

@tenderlove
Member

What version of Ruby are you using? Also, does it require that you use apache, or do you see the same leak on different web servers? FWIW, I'm running with trunk Ruby and using WEBrick in production mode. I see large swings in memory usage, but it isn't growing unbounded.

@rails-bot rails-bot added the stale label Aug 19, 2014
@rails-bot

This issue has been automatically marked as stale because it has not been commented on for at least
three months.

The resources of the Rails team are limited, and so we are asking for your help.

If you can still reproduce this error on the 4-1-stable, 4-0-stable branches or on master,
please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions.

@rails-bot rails-bot closed this Nov 19, 2014
@rails-bot

This issue has been automatically closed because of inactivity.

If you can still reproduce this error on the 4-1-stable, 4-0-stable branches or on master,
please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions.

@belov
belov commented Jul 21, 2015

I confirm this issue.
This https://github.com/belov/view_template_leak_app contain results some tests and increase memory chart.
Please see.

testable env:

  • ubuntu 14.04
  • ruby 2.2.2
  • rails 4.2.3
  • unicorn 4.9.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment