Memory leak under Rack::Test

[mattsnyder]

We've been writing our API using Grape and the DSL is great! However as we write more endpoints and test them with RSpec we've seen memory utilization skyrocket! Upwards of 8+G to test ~7 endpoints!

I wrote a spec file to try and isolate the problem and noticed three classes that with every call to an endpoint leave behind numerous objects that cannot be garbage collected.

Here is the spec I wrote that shows the increase in references for these classes:
Grape::Route, Grape::Endpoint,Grape::Util::HashStack,Regexp

And after 3 calls to the endpoint, I saw the reference counts go from

"Grape::Route 25 references"
"Grape::Endpoint 36 references"
"Grape::Util::HashStack 50 references"
"Regexp 4386 references"

to

"Grape::Route 47 references"
"Grape::Endpoint 47 references"
"Grape::Util::HashStack 60 references"
"Regexp 4687 references"

require 'spec_helper'

class GcApi < Grape::API
  prefix 'gcapi'
  format :json

  resources :testing do
    get do
      true
    end

    put do
      false
    end
  end
end

UserTesting::Orders::Application.routes.draw do
  mount GcApi => '/'
end


describe GcApi do
 include Rack::Test::Methods
 def app
   subject
 end

  def log(object, message)
    header = "*"*20 + message + "*"*20
    p header
    p object
    p "*"*header.length
  end

  def dumpit(file)
    counts = Hash.new{ 0 }
    ObjectSpace.each_object do |o|
      counts[o.class] += 1
    end
    file = File.open(file, 'w')
    counts.each do |key, val|
      file.puts(key.to_s + "=" + val.to_s)
    end
    file.close
  end

  def show_references(klass)
    count = 0
    ObjectSpace.each_object(klass) do |o|
      count += 1
    end
    p klass.to_s + " #{count} references"
  end

  def accept_header
    { 'HTTP_ACCEPT' => "application/json" }
  end

  before(:all) do
    GC.start
    log ObjectSpace.count_objects, "BEFOREALL"
    # dumpit("before.txt")
    show_references(Grape::Route)
    show_references(Grape::Endpoint)
    show_references(Grape::Util::HashStack)
    show_references(Regexp)
  end

  after(:all) do
    show_references(Grape::Route)
    show_references(Grape::Endpoint)
    show_references(Grape::Util::HashStack)
    show_references(Regexp)
    GC.start
    log ObjectSpace.count_objects, "AFTERALL"
    # dumpit("after.txt")
  end

  it "should run quickly" do #1635
    get "/gcapi/testing.json", nil, {}
    last_response.body.should == "true"
  end

  it "should run longer" do #1635
    get "/gcapi/testing.json", nil, {}
    last_response.should be_empty # false assumption
  end

  it "should run quickly or maybe not" do #409
    put "/gcapi/testing.json", nil, {}
    last_response.body.should == "false"
  end

  # it "should be able to GC" do
  #   GC.start
  #   sleep 5
  #   log ObjectSpace.count_objects, "BEFORESIMPLETEST"
  #   (1+1).should eql(2)
  #   GC.start
  #   sleep 5
  #   log ObjectSpace.count_objects, "AFTERSIMPLETEST"
  # end
end

[dblock]

Just to confirm, this is only happening under RSpec (Rack::Test)? See if you can port the above into Grape's RSpec tests (the above seems to need UserTesting::Orders::Application and I'd like to remove that one first.

[mattsnyder]

As far as I am aware, yes, only under RSpec. UserTesting::Orders::Application is just a Rails app

[mattsnyder]

@dblock I was able to recreate the scenario in the Grape repo's spec suite. See #667

[dblock]

Thanks @mattsnyder. Maybe you can also fix it? :)

[mattsnyder]

Noticing that each get request in a spec generates 4 un-collectable references to Grape::Endpoint.

For example, this test itself generates 4 references (extracted from the PR demonstrating this bug)

 context "when performing a single call against the endpoint" do
    it 'is adding an uncollectable reference to Grape::Endpoint' do
      GC.start
      pre_test_counts = count_references concerns

      get 'gcapi/testing'
      #expect(last_response.body).to eql "Hello World"

      GC.start
      post_test_counts = count_references concerns

      puts post_test_counts
      expect(post_test_counts).to eql pre_test_counts
    end
  end

I'm looking into the why, but any thoughts off the top of your head @dblock ?

[mattsnyder]

@dblock How does reset! work in the Grape::API class?
https://github.com/mattsnyder/grape/blob/master/lib/grape/api.rb#L566

I've not seen this approach used before and it appears this is maybe not releasing everything it should.

[dblock]

The reset! call is more like a setup, implemented here. If calling it explicitly fixes the RSpec issue, then maybe that's the way to go. However, it still seems fishy to me that a stable API would continue growing just because you call its methods :)

[mattsnyder]

It does not fix it. I think I see what #reset! is doing now; basically removing all the setup information after the endpoint has added each route via Rack::Mount::add_route.

What I did notice is that it adds a route for each HTTP verb (GET, POST, OPTIONS, etc) even though I've only defined GET. Do you know why that is? I don't think that is the problem, just curious as to why.

[mattsnyder]

Closing this issue. Found that the memory leak was related to mounting the API in a Rails app and then testing the Grape::API using Rack::Test::Methods in the specs which results in the endpoints recompiling for each test ...thus consuming large amounts of memory.

The uncollected references I observed in the linked PR were accurate since Grape::API is a singleton and a Global object so the actual instance will not get garbage collected.