Skip to content

Commit

Permalink
Publish.
Browse files Browse the repository at this point in the history
  • Loading branch information
mame committed Mar 31, 2016
0 parents commit 3567190
Show file tree
Hide file tree
Showing 64 changed files with 8,588 additions and 0 deletions.
23 changes: 23 additions & 0 deletions .dockerignore
@@ -0,0 +1,23 @@
.bundle/
Gemfile.lock
coverage/
doc/
pkg/
vendor/
optcarrot-*.gem

.git
.dockerignore
.*.sw*
**/.*.sw*
tools/nes-test-roms
SDL2.dll
video.png
video.gif
audio.wav
stackprof-*.dump
perf.data
perf.data.old
benchmark/bm-*.csv
benchmark/Dockerfile.*
tmp
21 changes: 21 additions & 0 deletions .gitignore
@@ -0,0 +1,21 @@
/.bundle/
/Gemfile.lock
/benchmark
/coverage/
/pkg/
/vendor/
optcarrot-*.gem

.*.sw*
/tools/nes-test-roms
video.png
video.gif
audio.wav
stackprof-*.dump
perf.data
perf.data.old
benchmark/bm-*.csv
ppu-core.rb
cpu-core.rb
benchmark/Dockerfile.*
benchmark/*-core-opt-*.rb
115 changes: 115 additions & 0 deletions .rubocop.yml
@@ -0,0 +1,115 @@
AllCops:
Exclude:
- "*-core.rb"
- "tmp/**"
- "test/**"
- "benchmark/*-core-opt-*.rb"
- "benchmark/Dockerfile.*"

# "eval" is the swiss army knife
Lint/Eval:
Enabled: false

# "while true" loop is easy and fast
Lint/LiteralInCondition:
Enabled: false
Lint/Loop:
Enabled: false
Style/InfiniteLoop:
Enabled: false

# `String#%` is a great invention of Ruby
Style/FormatString:
EnforcedStyle: percent

# I hate frozen string literal
Style/FrozenStringLiteralComment:
Enabled: false

# 10_000 looks dirty to me
Style/NumericLiterals:
MinDigits: 6

# explicit return is sometimes good for consistency
Style/RedundantReturn:
Enabled: false

# I like `foo {|x| bar(x) }` and `foo { bar }`
Style/SpaceInsideBlockBraces:
EnforcedStyleForEmptyBraces: space
SpaceBeforeBlockParameters: false

# `"foo#{bar}baz"` looks too dense to me
Style/SpaceInsideStringInterpolation:
EnforcedStyle: space

# I consistently use double quotes
Style/StringLiterals:
EnforcedStyle: double_quotes
Style/StringLiteralsInInterpolation:
EnforcedStyle: double_quotes

# A trailing comma in array literal is super cool
Style/TrailingCommaInLiteral:
Enabled: false

# I don't like this so much but virtually needed for ffi struct layout
Style/TrailingCommaInArguments:
Enabled: false

# I don't want to fill my code with `.freeze`
Style/MutableConstant:
Enabled: false

# I have no idea why this is prohibited...
Style/ParallelAssignment:
Enabled: false

# Backrefs are indeed dirty, but `Regexp.last_match` is too verbose
Style/PerlBackrefs:
Enabled: false

# I have forgotten `Kernel#fail`.
Style/SignalException:
EnforcedStyle: only_raise

# I think `{|ary| ary.size }` is not so bad since its type is explicit
Style/SymbolProc:
Enabled: false

# Wrapping a code is so bad? Case-by-case.
Style/GuardClause:
MinBodyLength: 5

# I use hyphen-separated case for script program.
Style/FileName:
Exclude:
- 'tools/*.rb'

# `'abc'.casecmp('ABC') == 0` looks very stupid to me
Performance/Casecmp:
Enabled: false

# I don't care class/module size
Metrics/ClassLength:
Max: 1000
Metrics/ModuleLength:
Max: 1000

# I accept two-screen size (about 100 lines?)
Metrics/MethodLength:
Max: 100

# Don't worry, my terminal is big enough
Metrics/LineLength:
Max: 120

# Code metrics is good, but I think the default is too strict...
Metrics/CyclomaticComplexity:
Max: 15
Metrics/PerceivedComplexity:
Max: 20
Metrics/AbcSize:
Max: 100
Metrics/BlockNesting:
Max: 5
3 changes: 3 additions & 0 deletions Gemfile
@@ -0,0 +1,3 @@
source "https://rubygems.org"

gem "ffi"
48 changes: 48 additions & 0 deletions README.md
@@ -0,0 +1,48 @@
# Optcarrot: A NES Emulator for Ruby Benchmark

## Project Goals

This project aims to provide an "enjoyable" benchmark for Ruby implementation to drive ["Ruby3x3: Ruby 3 will be 3 times faster"][ruby3x3].

The specific target is a NES (Nintendo Entertainment System) emulator that works at *20 fps* in Ruby 2.0. An original NES works at 60 fps. If Ruby3x3 is succeeded, we can enjoy a NES game with Ruby!

NOTE: We do *not* aim to create a practical NES emulator. There have been already many great emulators available. We recommend you use another emulator if you just want to play a game.

## Basic usage

SDL2 is required.

$ git clone http://github.com/mame/optcarrot.git
$ cd optcarrot
$ bin/optcarrot examples/Lan_Master.nes

|key |button |
|------|-------------|
|arrow |D-pad |
|`Z` |A button |
|`X` |B button |
|space |Start button |
|return|Select button|

See [`doc/bonus.md`](doc/bonus.md) for advanced usage.

## Benchmark example

![doc/benchmark-default.png]

See [`doc/benchmark.md`](doc/benchmark.md) for the measurement condition.

## Optimized mode

It may run faster with the option `--opt`.

$ bin/optcarrot --opt examples/Lan_Master.nes

This option will generate an optimized (and super-dirty) Ruby code internally, and replace some bottleneck methods with them. See [`doc/internal.md`](doc/internal.md) in detail.

## Acknowledgement

We appreciate all the people who devoted efforts to NES analysis. If it had not been not for the [NESdev Wiki][nesdev-wiki], we could not create this program. We also read the source code of Nestopia, NESICIDE, and others. We used the test ROMs due to NESICIDE.

[ruby3x3]: https://www.youtube.com/watch?v=LE0g2TUsJ4U&t=3248
[nesdev-wiki]: http://wiki.nesdev.com/w/index.php/NES_reference_guide
18 changes: 18 additions & 0 deletions Rakefile
@@ -0,0 +1,18 @@
task :test do
ruby "tools/run-tests.rb"
end

task :benchmark do
ruby "tools/run-benchmark.rb", "all", "-m", "all", "-c", "10"
end

task :wc do
puts "lines of minimal source code:"
sh "wc -l bin/optcarrot lib/optcarrot.rb lib/optcarrot/*.rb"
end

task :"wc-all" do
sh "wc -l bin/optcarrot lib/optcarrot.rb lib/optcarrot/*.rb lib/optcarrot/*/*.rb"
end

task default: :test
6 changes: 6 additions & 0 deletions bin/optcarrot
@@ -0,0 +1,6 @@
#!/usr/bin/env ruby

# I'm too lazy to type `-Ilib` every time...
require_relative "../lib/optcarrot"

Optcarrot::NES.new.run
Binary file added doc/benchmark-default.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/benchmark-optimized.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
99 changes: 99 additions & 0 deletions doc/benchmark.md
@@ -0,0 +1,99 @@
# Ruby implementation benchmark with Optcarrot

![doc/benchmark-optimized.png]

## Experimental conditions

* Core i7 4500U (1.80GHz) / Ubuntu 15.10
* Command: `ruby -v -Ilib bin/optcarrot -r./tools/shim --benchmark examples/Lan_Master.nes`
* This runs the first 180 frames (three seconds), and prints the fps of the last ten frames.
* `--benchmark` mode implies no GUI, so GUI overhead is not included.
* [`tools/shim.rb`](tools/shim.rb) is required for incompatibility of Ruby implementations.
* `--opt` option is added for the optimized mode.
* Measured fps 10 times for each, and calculated the average over the runs.
* The error bars represent the standard deviation.

## Ruby implementations

* ruby23: `ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]`
* ruby22: `ruby 2.2.4p230 (2015-12-16 revision 53155) [x86_64-linux]`
* ruby21: `ruby 2.1.8p440 (2015-12-16 revision 53160) [x86_64-linux]`
* ruby20: `ruby 2.0.0p648 (2015-12-16 revision 53162) [x86_64-linux]`
* ruby193: `ruby 1.9.3p551 (2014-11-13 revision 48407) [x86_64-linux]`
* ruby187: `ruby 1.8.7 (2013-06-27 patchlevel 374) [x86_64-linux]`

* omrpreview: `ruby 2.2.3p97 (OMR Preview r1) (2015-04-14) [x86_64-linux]`
* `OMR_JIT_OPTIONS='-Xjit'` is specified.

* jruby9k: `jruby 9.0.5.0 (2.2.3) 2016-01-26 7bee00d OpenJDK 64-Bit Server VM 25.72-b15 on 1.8.0_72-internal-b15 +jit [linux-amd64]`

* jruby17: `jruby 1.7.24 (1.9.3p551) 2016-01-20 bd68d85 on OpenJDK 64-Bit Server VM 1.8.0_72-internal-b15 +jit [linux-amd64]`
* `--server` is specified.

* rubinius: `rubinius 3.19 (2.2.2 1cc41ddc 2016-02-27 3.4 JI) [x86_64-linux-gnu]`

* mruby: `mruby 1.2.0 (2015-11-17)`
* Patched so that `Fixnum#/` returns an Integer instead of Float.

* topaz: `topaz (ruby-1.9.3p125) (git rev 019daf0) [x86_64-linux]`
* Failed to run the optimized mode maybe because the generated core is so large.

* opal: `Opal v0.10.0.dev`
* Failed to run the default mode because of lack of Fiber.

See [`tools/run-benchmark.rb`](tools/run-benchmark.rb) for the actual commands.

## Remarks

This benchmark may not be fair inherently. Optcarrot is somewhat tuned for MRI since I developed it with MRI.

The optimized mode assumes that case statement is implemented with "jump table" if all `when` clauses have trivial immediate values such as Integer. This is true for MRI, but it is known that [JRuby 9k](https://github.com/jruby/jruby/issues/3672) and [Rubinius](https://github.com/rubinius/rubinius-code/issues/2) are not (yet). OMR preview also seems not to support JIT for `opt_case_dispatch` instruction.

## Hints for Ruby implementation developers

* This program is purely CPU-intensive. Any improvement of I/O and GC will not help.

* As said in remarks, this program assumes that the implementation will optimize `case` statements by "jump-table". Checking each clauses in order will be too slow.
* Implementation note: In the optimized mode (`--opt` option), CPU/PPU evaluators consist of one loop with a big `case` statement dispatching upon the current opcode or clock.

* The hotspot is `PPU#run` and `CPU#run`. The optimized mode replaces them with an automatically generated and optimized source code by using `eval`.
* You can see the generated code with `--dump-cpu` and `--dump-ppu`. See also [`doc/internal.md`](doc/internal.md).

* The hotspot uses no reflection-like features except `send` and `Method#[]`.
* Implementation note: CPU dispatching uses `send` in the default mode. Memory-mapped I/O is implemented by exploiting polymorphism of `Method#[]` and `Array#[]`.

* If you are a MRI developer, you can reduce compile time by using `miniruby`.

$ git clone https://github.com/ruby/ruby.git
$ cd ruby
$ ./configure
$ make miniruby -j 4
$ ./miniruby /path/to/optcarrot --benchmark /path/to/Lan_Master.nes

## How to benchmark
### How to use optcarrot as a benchmark

With `--benchmark` option, Optcarrot works in the headless mode (i.e., no GUI), run a ROM in the first 180 frames, and prints the fps of the last ten frames.

$ /path/to/ruby bin/optcarrot --benchmark examples/Lan_Master.nes
fps: 26.74081335620352
checksum: 59662

By default, Optcarrot depends upon [ffi][ffi] gem. The headless mode has *zero* dependency: no gems, no external libraries, even no stdlib are required. Unfortunately, you need to use [`tools/shim.rb`](tools/shim.rb) due to some incompatibilities between MRI and other implementations.

$ jruby -r ./tools/shim.rb -Ilib bin/optcarrot --benchmark examples/Lan_Master.nes

### How to run the full benchmark

This script will build docker images for some Ruby implementations, run a benchmark on them, and create `benchmark/bm-latest.csv`.

$ ruby tools/run-benchmark.rb all -m all -c 10

Note that it will take a few hours. If you want to specify target, do:

$ ruby tools/run-benchmark.rb ruby23 -m all

If you want to try [rubyomr-preview][omr], you need to load its docker image before running the benchmark.

[ffi]: http://rubygems.org/gems/ffi
[omr]: https://github.com/rubyomr-preview/rubyomr-preview

0 comments on commit 3567190

Please sign in to comment.