ObjectSpace.each_object is slow #1064

Closed
DAddYE opened this Issue Jul 5, 2011 · 6 comments

Comments

Projects
None yet
3 participants
@DAddYE

DAddYE commented Jul 5, 2011

As reported w/twitter running my tests on rubinius take much much much time than a vanilla 1.8.7 or 1.9.2, I can't figure way.

I will report here the smallest test suite but consider that this: https://github.com/padrino/padrino-integration require 2/3 hours instead of 20/30min on 1.8.7,1.9.2, REE.

Test suite that I use now is: https://github.com/padrino/padrino-framework

# Commands
$ bundle install
$ cd padrino-core
$ rake test

Results:

# Ruby 1.9.2
Finished in 5.899506 seconds.

# Ruby 1.8.7
Finished in Finished in 8.206628 seconds.

# rubinius 1.2.4dev (1.8.7 a8890734 yyyy-mm-dd JI) [x86_64-apple-darwin11.0.0]
Finished in 35.791055 seconds.

Tried also with:

$ RBX_OPT=-Xprofile rake test
/Users/DAddYE/.rvm/rubies/rbx-head/bin/rbx -I"lib:test" -I"/Users/DAddYE/.rvm/gems/rbx-head/gems/rake-0.9.2/lib" "/Users/DAddYE/.rvm/gems/rbx-head/gems/rake-0.9.2/lib/rake/rake_test_loader.rb" "test/**/test_*.rb" 
Loaded suite /Users/DAddYE/.rvm/gems/rbx-head/gems/rake-0.9.2/lib/rake/rake_test_loader
Started
.....................................................................................................................................................................................
Finished in 35.791055 seconds.

181 tests, 619 assertions, 0 failures, 0 errors

Tested on osx and centos, installation through rvm.

Any ideas?

@dbussink

This comment has been minimized.

Show comment Hide comment
@dbussink

dbussink Jul 6, 2011

Member

A lot of time in the tests is used in ObjectSpace. If you can, please try to not use ObjectSpace since it will always be very slow on Rubinius. Here's a flat profile of the data:

https://gist.github.com/0b36ec30926578ddc5b8

If you want, I can send you a HTML version of the complete profiling data. If you look at that, you can see it spends mosts of the time in Padrino's reloading code (which uses ObjectSpace and some class_eval stuff too I found: https://gist.github.com/e0992719d4ee7ff32bec)

We can't really make ObjectSpace any further and really encourage people to look for alternative solutions that don't need that.

BTW, the reason for not seeing a profile is because you used RBX_OPT and not RBXOPT. The latter is the option needed to pass it into Rubinius.

Member

dbussink commented Jul 6, 2011

A lot of time in the tests is used in ObjectSpace. If you can, please try to not use ObjectSpace since it will always be very slow on Rubinius. Here's a flat profile of the data:

https://gist.github.com/0b36ec30926578ddc5b8

If you want, I can send you a HTML version of the complete profiling data. If you look at that, you can see it spends mosts of the time in Padrino's reloading code (which uses ObjectSpace and some class_eval stuff too I found: https://gist.github.com/e0992719d4ee7ff32bec)

We can't really make ObjectSpace any further and really encourage people to look for alternative solutions that don't need that.

BTW, the reason for not seeing a profile is because you used RBX_OPT and not RBXOPT. The latter is the option needed to pass it into Rubinius.

@DAddYE

This comment has been minimized.

Show comment Hide comment
@DAddYE

DAddYE Jul 13, 2011

Thanks @dbussink for the explanation and sorry for RBX_OPT, I will try to try a better way to traverse ObjectSpace. Thanks!

DAddYE commented Jul 13, 2011

Thanks @dbussink for the explanation and sorry for RBX_OPT, I will try to try a better way to traverse ObjectSpace. Thanks!

@DAddYE

This comment has been minimized.

Show comment Hide comment
@DAddYE

DAddYE Jul 13, 2011

@dbussink basically I hack (in a way that I hate) ObjectSpace due to a problem with rb tables where (from irb):

>> Foo = Class.new
=> Foo
>> ObjectSpace.each_object(Class).find { |c| c.name == 'Foo' }
=> Foo

That's correct... my problem on MRI is that:

>> Object.send(:remove_const, :Foo)
=> Foo
>> Object.const_defined?(:Foo)
=> false
>> ObjectSpace.each_object(Class).find { |c| c.name == 'Foo' }
=> Foo

As you can see Foo wasn't removed from ObjectSpace and to workaround I added my bad hack: Class.class_eval { klass } rescue false

DAddYE commented Jul 13, 2011

@dbussink basically I hack (in a way that I hate) ObjectSpace due to a problem with rb tables where (from irb):

>> Foo = Class.new
=> Foo
>> ObjectSpace.each_object(Class).find { |c| c.name == 'Foo' }
=> Foo

That's correct... my problem on MRI is that:

>> Object.send(:remove_const, :Foo)
=> Foo
>> Object.const_defined?(:Foo)
=> false
>> ObjectSpace.each_object(Class).find { |c| c.name == 'Foo' }
=> Foo

As you can see Foo wasn't removed from ObjectSpace and to workaround I added my bad hack: Class.class_eval { klass } rescue false

@DAddYE

This comment has been minimized.

Show comment Hide comment
@DAddYE

DAddYE Jul 13, 2011

I report that I've same result on RBX.

DAddYE commented Jul 13, 2011

I report that I've same result on RBX.

@evanphx

This comment has been minimized.

Show comment Hide comment
@evanphx

evanphx Nov 17, 2011

Member

Why do you need to remove the constant and then go back and retrieve it by name? If you know when it's removed, can't you stash a reference to it somewhere so you don't have to use ObjectSpace?

Member

evanphx commented Nov 17, 2011

Why do you need to remove the constant and then go back and retrieve it by name? If you know when it's removed, can't you stash a reference to it somewhere so you don't have to use ObjectSpace?

@dbussink

This comment has been minimized.

Show comment Hide comment
@dbussink

dbussink Nov 11, 2012

Member

Performance of object space isn't something actionable for us atm, so closing this one.

Member

dbussink commented Nov 11, 2012

Performance of object space isn't something actionable for us atm, so closing this one.

@dbussink dbussink closed this Nov 11, 2012

@vencha90 vencha90 referenced this issue in pry/pry Oct 9, 2014

Open

`ls` is slow on Rubinius #1133

brixen added a commit that referenced this issue Dec 11, 2014

Updated rubygems to 2.4.5.
RubyGems 2.4.5 installed

=== 2.4.5 / 2014-12-03

Bug fixes:

* Improved speed of requiring gems.  (Around 25% for a 60 gem test).  Pull
  request #1060 by unak.
* RubyGems no longer attempts to look up gems remotely with the --local flag.
  Pull request #1084 by Jeremy Evans.
* Executable stubs use the correct gem version when RUBYGEMS_GEMDEPS is
  active.  Issue #1072 by Michael Kaiser-Nyman.
* Fixed handling of pinned gems in lockfiles with versions.  Issue #1078 by
  Ian Ker-Seymer.
* Fixed handling of git@example:gem.git URIs.  Issue #1054 by Mogutan Mogu.
* Fixed handling of platforms retrieved from the dependencies API.  Issue
  #1058 and patch suggestion by tux-mind.
* RubyGems now suggests a copy-pasteable `gem pristine` command when
  extensions are missing.  Pull request #1057 by Shannon Skipper.
* Improved errors for long file names when packaging.  Pull request #1016 by
  Piotrek Bator.
* `gem pristine` now skips gems cannot be found remotely.  Pull request #1064
  by Tuomas Kareinen.
* `gem pristine` now caches gems to the proper directory.  Pull request #1064
  by Tuomas Kareinen.
* `gem pristine` now skips bundled gems properly.  Pull request #1064 by
  Tuomas Kareinen.
* Improved interoperability of Vagrant with RubyGems.  Pull request #1057 by
  Vít Ondruch.
* Renamed CONTRIBUTING to CONTRIBUTING.rdoc to allow markup.  Pull request
  #1090 by Roberto Miranda.
* Switched from #partition to #reject as only one collection is used.  Pull
  request #1074 by Tuomas Kareinen.
* Fixed installation of gems on systems using memory-mapped files.  Pull
  request #1038 by Justin Li.
* Fixed bug in Gem::Text#min3 where `a == b < c`.  Pull request #1026 by
  fortissimo1997.
* Fixed uninitialized variable warning in BasicSpecification.  Pull request
  #1019 by Piotr Szotkowski.
* Removed unneeded exception handling for cyclic dependencies.  Pull request
  #1043 by Jens Wille.
* Fixed grouped expression warning.  Pull request #1081 by André Arko.
* Fixed handling of platforms when writing lockfiles.

=== 2.4.4 / 2014-11-12

brixen added a commit that referenced this issue Apr 18, 2015

Updated RubyGems to 2.4.6.
RubyGems 2.4.6 installed

=== 2.4.6 / 2014-02-05

Bug fixes:

* Fixed resolving gems with both upper and lower requirement boundaries.
  Issue #1141 by Jakub Jirutka.
* Moved extension directory after require_paths to fix missing constant bugs
  in some gems with C extensions.  Issue #784 by André Arko, pull request
  #1137 by Barry Allard.
* Use Gem::Dependency#requirement when adding a dependency to an existing
  dependency instance.  Pull request #1101 by Josh Cheek.
* Fixed warning of shadowed local variable in Gem::Specification.  Pull request
  #1109 by Rohit Arondekar
* Gem::Requirement should always sort requirements before coercion to Hash.
  Pull request #1139 by Eito Katagiri.
* The `gem open` command should change the current working directory before
  opening the editor.  Pull request #1142 by Alex Wood.
* Ensure quotes are stripped from the Windows launcher script used to install
  gems.  Pull request #1115 by Youngjun Song.
* Fixed errors when writing to NFS to to 0444 files.  Issue #1161 by Emmanuel
  Hadoux.
* Removed dead code in Gem::StreamUI.  Pull request #1117 by mediaslave24.
* Fixed typos.  Pull request #1096 by hakeda.
* Relaxed CMake dependency for RHEL 6 and CentOS 6.  Pull request #1124 by Vít
  Ondruch.
* Relaxed Psych dependency.  Pull request #1128 by Vít Ondruch.

=== 2.4.5 / 2014-12-03

Bug fixes:

* Improved speed of requiring gems.  (Around 25% for a 60 gem test).  Pull
  request #1060 by unak.
* RubyGems no longer attempts to look up gems remotely with the --local flag.
  Pull request #1084 by Jeremy Evans.
* Executable stubs use the correct gem version when RUBYGEMS_GEMDEPS is
  active.  Issue #1072 by Michael Kaiser-Nyman.
* Fixed handling of pinned gems in lockfiles with versions.  Issue #1078 by
  Ian Ker-Seymer.
* Fixed handling of git@example:gem.git URIs.  Issue #1054 by Mogutan Mogu.
* Fixed handling of platforms retrieved from the dependencies API.  Issue
  #1058 and patch suggestion by tux-mind.
* RubyGems now suggests a copy-pasteable `gem pristine` command when
  extensions are missing.  Pull request #1057 by Shannon Skipper.
* Improved errors for long file names when packaging.  Pull request #1016 by
  Piotrek Bator.
* `gem pristine` now skips gems cannot be found remotely.  Pull request #1064
  by Tuomas Kareinen.
* `gem pristine` now caches gems to the proper directory.  Pull request #1064
  by Tuomas Kareinen.
* `gem pristine` now skips bundled gems properly.  Pull request #1064 by
  Tuomas Kareinen.
* Improved interoperability of Vagrant with RubyGems.  Pull request #1057 by
  Vít Ondruch.
* Renamed CONTRIBUTING to CONTRIBUTING.rdoc to allow markup.  Pull request
  #1090 by Roberto Miranda.
* Switched from #partition to #reject as only one collection is used.  Pull
  request #1074 by Tuomas Kareinen.
* Fixed installation of gems on systems using memory-mapped files.  Pull
  request #1038 by Justin Li.
* Fixed bug in Gem::Text#min3 where `a == b < c`.  Pull request #1026 by
  fortissimo1997.
* Fixed uninitialized variable warning in BasicSpecification.  Pull request
  #1019 by Piotr Szotkowski.
* Removed unneeded exception handling for cyclic dependencies.  Pull request
  #1043 by Jens Wille.
* Fixed grouped expression warning.  Pull request #1081 by André Arko.
* Fixed handling of platforms when writing lockfiles.

------------------------------------------------------------------------------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment