Add SecureRandom from Ruby 1.9.2-p290 #1111

Closed
wants to merge 723 commits into
from

Conversation

Projects
None yet
8 participants
Member

IPGlider commented Aug 11, 2011

This version of SecureRandom adds #urlsafe_base64, #uuid and Win32 support.

Brian Ford and others added some commits Jun 3, 2011

Brian Ford
Finally, we get to define ! and !=.
Historical Note: Long, long ago, ! and != were methods in Rubinius.
True story, not a fable. Evan had added them to Sydney parser. We
removed them because ruby_parser choked on them, hence the argument
was made that they were incompatible.
Brian Ford
String#[] returns a String on 1.9.
This is a quick hack. Lots of work still to do on encodings.
Brian Ford
Enable building with MRI or Rubinius 1.2.
This changes from building a bootstrap Melbourne specifically for
MRI to building a bootstrap Melbourne for Rubinius or MRI. In order
to build the 1.9 kernel, we must use the 1.9 Melbourne and compiler.
This requires a Rubinius bridge to load the compiler over the top
of the compiler already loaded.
Merge branch 'master' into 2.0
Conflicts:
	kernel/bootstrap/thread.rb
	lib/ext/melbourne/Rakefile
	vm/builtin/thread.cpp
	vm/configuration.hpp
	vm/environment.hpp
	vm/objectmemory.cpp
	vm/util/immix.hpp
Merge branch 'master' into 2.0
Conflicts:
	web/_posts/2011-06-07-inside-rubinius-20-preview.markdown

Brian Ford and others added some commits Aug 1, 2011

Brian Ford
Improve distribution of Object#hash for immediates.
The primary reason for this is better Hash preformance. The following
benchmarks compare 1.8 Hash, 1.9 Hash and Hash Array Mapped Trie insertion
performance for Symbol and String-keyed Hashes.

*** before changing Object#hash ***

$ bin/benchmark -tx benchmark/rubinius/hash/bench_insert.rb
=== bin/rbx ===
       insert sym 10    45316.1 (±2.9%) i/s -     228452 in   5.046476s (cycle=2786)
      insert sym 100     3564.5 (±1.6%) i/s -      18084 in   5.074744s (cycle=274)
     insert sym 1000      370.9 (±1.1%) i/s -       1850 in   4.988637s (cycle=37)
    insert sym 10000       31.4 (±6.4%) i/s -        156 in   4.991863s (cycle=3)
insert sym existing 10
                        90670.7 (±1.0%) i/s -     454875 in   5.017297s (cycle=3639)
insert sym existing 10000
                           55.8 (±7.2%) i/s -        279 in   5.025598s (cycle=3)
       insert str 10    46252.2 (±1.3%) i/s -     234213 in   5.064731s (cycle=4109)
      insert str 100     4334.6 (±1.5%) i/s -      21892 in   5.051767s (cycle=421)
     insert str 1000      458.7 (±1.3%) i/s -       2295 in   5.004059s (cycle=45)
    insert str 10000       31.1 (±6.4%) i/s -        156 in   5.053014s (cycle=3)
insert str existing 10
                       121534.9 (±1.9%) i/s -     606864 in   4.995140s (cycle=4304)
insert str existing 10000
                           61.2 (±8.2%) i/s -        303 in   4.980165s (cycle=3)

$ bin/benchmark -t 'rbx -X19' benchmark/rubinius/hash/bench_insert.rb
=== rbx -X19 ===
       insert sym 10    33796.3 (±0.8%) i/s -     170940 in   5.058286s (cycle=2220)
      insert sym 100     2676.6 (±0.8%) i/s -      13462 in   5.029890s (cycle=254)
     insert sym 1000      286.3 (±1.4%) i/s -       1428 in   4.988640s (cycle=28)
    insert sym 10000       24.5 (±4.1%) i/s -        122 in   5.001505s (cycle=2)
insert sym existing 10
                        67456.0 (±1.4%) i/s -     337065 in   4.997809s (cycle=2931)
insert sym existing 10000
                           35.7 (±8.4%) i/s -        176 in   4.989022s (cycle=2)
       insert str 10    35706.9 (±1.4%) i/s -     178872 in   5.010427s (cycle=3084)
      insert str 100     3336.1 (±0.9%) i/s -      16744 in   5.019439s (cycle=322)
     insert str 1000      354.0 (±1.7%) i/s -       1785 in   5.044361s (cycle=35)
    insert str 10000       24.5 (±8.2%) i/s -        122 in   5.020168s (cycle=2)
insert str existing 10
                       104821.4 (±2.4%) i/s -     522327 in   4.985866s (cycle=2951)
insert str existing 10000
                           39.7 (±10.1%) i/s -        196 in   5.003198s (cycle=2)

$ bin/benchmark -t 'rbx -X19 -Xhash.hamt' benchmark/rubinius/hash/bench_insert.rb
=== rbx -X19 -Xhash.hamt ===
       insert sym 10    51378.1 (±1.8%) i/s -     257656 in   5.016663s (cycle=2408)
      insert sym 100     3719.7 (±1.4%) i/s -      18761 in   5.044790s (cycle=257)
     insert sym 1000      287.8 (±1.0%) i/s -       1456 in   5.058938s (cycle=28)
    insert sym 10000       29.6 (±0.0%) i/s -        148 in   5.005240s (cycle=2)
insert sym existing 10
                       198151.1 (±1.8%) i/s -     989166 in   4.993838s (cycle=4021)
insert sym existing 10000
                           39.4 (±2.5%) i/s -        198 in   5.026739s (cycle=2)
       insert str 10    43397.1 (±1.2%) i/s -     219120 in   5.049999s (cycle=3652)
      insert str 100     3639.3 (±0.9%) i/s -      18179 in   4.995538s (cycle=343)
     insert str 1000      271.1 (±1.5%) i/s -       1377 in   5.081294s (cycle=27)
    insert str 10000       20.6 (±4.9%) i/s -        104 in   5.073148s (cycle=2)
insert str existing 10
                       170525.0 (±2.8%) i/s -     849387 in   4.985577s (cycle=3677)
insert str existing 10000
                           20.2 (±5.0%) i/s -        101 in   5.029512s (cycle=1)

*** after changing Object#hash ***

$ bin/benchmark -tx benchmark/rubinius/hash/bench_insert.rb
=== bin/rbx ===
       insert sym 10    56719.7 (±1.5%) i/s -     283836 in   5.005428s (cycle=3379)
      insert sym 100     4915.8 (±1.6%) i/s -      24618 in   5.009323s (cycle=373)
     insert sym 1000      513.2 (±1.0%) i/s -       2600 in   5.066852s (cycle=50)
    insert sym 10000       38.7 (±7.8%) i/s -        192 in   4.998104s (cycle=3)
insert sym existing 10
                       145823.5 (±1.9%) i/s -     726950 in   4.987029s (cycle=4690)
insert sym existing 10000
                           74.0 (±8.1%) i/s -        366 in   4.977395s (cycle=3)
       insert str 10    46983.6 (±1.3%) i/s -     236094 in   5.025931s (cycle=4142)
      insert str 100     4419.2 (±0.9%) i/s -      22366 in   5.061532s (cycle=422)
     insert str 1000      459.1 (±1.3%) i/s -       2295 in   4.999759s (cycle=45)
    insert str 10000       31.2 (±9.6%) i/s -        156 in   5.040123s (cycle=3)
insert str existing 10
                       120257.5 (±1.7%) i/s -     602009 in   5.007466s (cycle=4331)
insert str existing 10000
                           61.6 (±8.1%) i/s -        306 in   5.000727s (cycle=3)

$ bin/benchmark -t 'rbx -X19' benchmark/rubinius/hash/bench_insert.rb=== rbx -X19 ===
       insert sym 10    44896.0 (±2.0%) i/s -     225910 in   5.033973s (cycle=2755)
      insert sym 100     4099.4 (±2.9%) i/s -      20631 in   5.037513s (cycle=299)
     insert sym 1000      436.4 (±3.4%) i/s -       2193 in   5.032128s (cycle=43)
    insert sym 10000       32.3 (±6.2%) i/s -        162 in   5.035883s (cycle=3)
insert sym existing 10
                       111895.6 (±3.1%) i/s -     560280 in   5.012236s (cycle=3864)
insert sym existing 10000
                           60.8 (±8.2%) i/s -        303 in   5.021569s (cycle=3)
       insert str 10    36363.7 (±1.5%) i/s -     182913 in   5.031297s (cycle=3209)
      insert str 100     3377.3 (±1.4%) i/s -      16881 in   4.999319s (cycle=331)
     insert str 1000      361.6 (±1.7%) i/s -       1836 in   5.079493s (cycle=36)
    insert str 10000       24.5 (±8.2%) i/s -        122 in   5.025431s (cycle=2)
insert str existing 10
                        99391.7 (±1.9%) i/s -     496188 in   4.994262s (cycle=3222)
insert str existing 10000
                           39.8 (±10.1%) i/s -        196 in   4.996934s (cycle=2)

$ bin/benchmark -t 'rbx -X19 -Xhash.hamt' benchmark/rubinius/hash/bench_insert.rb
=== rbx -X19 -Xhash.hamt ===
       insert sym 10    67406.1 (±1.0%) i/s -     338247 in   5.018569s (cycle=3717)
      insert sym 100     4252.9 (±1.7%) i/s -      21329 in   5.016802s (cycle=277)
     insert sym 1000      301.2 (±1.7%) i/s -       1508 in   5.007770s (cycle=29)
    insert sym 10000       24.0 (±0.0%) i/s -        120 in   5.005557s (cycle=2)
insert sym existing 10
                       278739.8 (±2.1%) i/s -    1389535 in   4.987352s (cycle=5365)
insert sym existing 10000
                           34.7 (±2.9%) i/s -        174 in   5.019634s (cycle=2)
       insert str 10    40866.2 (±1.0%) i/s -     206160 in   5.045310s (cycle=3436)
      insert str 100     3612.7 (±1.0%) i/s -      18306 in   5.067711s (cycle=339)
     insert str 1000      267.8 (±1.1%) i/s -       1352 in   5.049568s (cycle=26)
    insert str 10000       20.1 (±9.9%) i/s -        100 in   5.021534s (cycle=1)
insert str existing 10
                       162265.4 (±2.2%) i/s -     810117 in   4.995049s (cycle=3507)
insert str existing 10000
                           29.0 (±6.9%) i/s -        144 in   5.004102s (cycle=2)

There is another way to see the effect of this change using the
Hash::Inspectors from the previous commit.

These IRB sessions are after the change:

$ rbx -X19 -r ./resources/experiments/hash/code/bucket_inspector.rb -r ./benchmark/rubinius/hash/setup.rb
irb(main):001:0> h = new_sym_hash 10000; nil
=> nil
irb(main):002:0> i = Hash::Inspector.new h; nil
=> nil
irb(main):003:0> puts i.report
Hash::Inspector report for 4064

Items:     10000
Bytes:     439640
Depth:     5
Table:     16384
Occupancy: 45.67%

Depth Stats
  Max:     1 (5416)
  Min:     5 (4)
  Median:  3 (350)
  Mean:    1

$ rbx -X19 -Xhash.hamt -r ./resources/experiments/hash/code/hamt_inspector.rb -r ./benchmark/rubinius/hash/setup.rb
irb(main):001:0> h = new_sym_hash 10000; nil
=> nil
irb(main):002:0> i = Hash::Inspector.new h; nil
=> nil
irb(main):003:0> puts i.report
Hash::Inspector report for 3728

Items:     10000
Bytes:     816872
Depth:     4
Tries:     3146
Occupancy: 100.00%

Depth Stats
  Max:     2 (2894)
  Min:     4 (8)
  Median:  3 (180)
  Mean:    2

Branching Stats
  Max:     2 (1276)
  Min:     9 (1)
  Median:  57 (11)
  Mean:    4

These are the reports from Hash::Inspector before the change:

1. HAMT: 10,000 elt Symbol-keyed

Hash::Inspector report for 3852

Items:     10000
Bytes:     669816
Depth:     2
Tries:     520
Occupancy: 12.50%

Depth Stats
  Max:     2 (512)
  Min:     1 (8)
  Median:  2 (512)
  Mean:    1

Branching Stats
  Max:     20 (272)
  Min:     64 (8)
  Median:  19 (240)
  Mean:    20

2. HAMT: 10,000 elt String-keyed

Hash::Inspector report for 4176

Items:     10000
Bytes:     1456088
Depth:     4
Tries:     3132
Occupancy: 100.00%

Depth Stats
  Max:     2 (2868)
  Min:     4 (2)
  Median:  3 (198)
  Mean:    2

Branching Stats
  Max:     2 (1313)
  Min:     10 (1)
  Median:  57 (7)
  Mean:    4

3. Bucket: 10,000 elt Symbol-keyed

Hash::Inspector report for 4048

Items:     10000
Bytes:     439752
Depth:     5
Table:     16384
Occupancy: 12.50%

Depth Stats
  Max:     5 (1808)
  Min:     4 (240)
  Median:  5 (1808)
  Mean:    4

4. Bucket: 10,000 elt String-keyed

Hash::Inspector report for 4208

Items:     10000
Bytes:     686472
Depth:     5
Table:     16384
Occupancy: 45.54%

Depth Stats
  Max:     1 (5420)
  Min:     5 (12)
  Median:  3 (360)
  Mean:    1

Note that the better Object#hash distribution significantly improves occupancy
of the root table, shortening the average depth that must be probed in both
implementations of Hash. However, this distribution also resulted in a greater
max depth for HAMT.
Contributor

headius commented on b64326a Aug 3, 2011

Perf of HAMT versus 1.8 hash seems a bit of a mixed bag, yeah? I guess it's probably also doing insertion ordering, though, so that's a solid gain. Any idea what memory-efficiency of HAMT versus 1.8 or 1.9 hash looks like?

Contributor

headius replied Aug 3, 2011

FWIW, a quick eyeballing of process size for inserting 1M new hash instances of one key/value into an array shows the size is about the same for 1.8 hash and HAMT. That's definitely nice. I don't know how to get a proper dump of actual memory used, though.

dbussink and others added some commits Aug 3, 2011

Merge remote-tracking branch 'origin/master' into 2.0.0pre
Conflicts:
	kernel/bootstrap/rubinius.rb
	vm/builtin/system.cpp
	vm/builtin/system.hpp
	vm/signal.cpp
	vm/signal.hpp
Merge remote-tracking branch 'origin/master' into 2.0.0pre
Conflicts:
	configure
	vm/builtin/system.cpp
Add SecureRandom from Ruby 1.9.2-p290
This version adds #urlsafe_base64, #uuid and Win32 support.
Member

IPGlider commented Aug 11, 2011

Closing.

@IPGlider IPGlider closed this Aug 11, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment