using:
Please note that these are mostly synthetic benchmarks!
| Benchmark | MRI | YARV | JRuby | Rubinius | 1st | 2nd | 3rd | (in seconds) |
| app_answer | 0.61 | 0.14 | 1.00 | 0.22 | YARV | Rubinius | MRI | |
| app_erb | 2.18 | 2.10 | 7.30 | error | YARV | MRI | JRuby | |
| app_factorial | error | 1.09 | 4.55 | 1.51 | YARV | Rubinius | JRuby | |
| app_fib | 7.93 | 1.96 | 7.60 | 3.72 | YARV | Rubinius | JRuby | |
| app_mandelbrot | 2.76 | 1.23 | 3.50 | 21.35 | YARV | MRI | JRuby | |
| app_pentomino | 114.43 | 59.07 | 181.93 | t/o | YARV | MRI | JRuby | |
| app_raise | 2.85 | 3.42 | 2.65 | 3.24 | JRuby | MRI | Rubinius | |
| app_strconcat | 1.90 | 1.42 | 1.80 | 1.80 | YARV | JRuby | Rubinius | |
| app_tak | 9.57 | 2.64 | 8.55 | 4.80 | YARV | Rubinius | JRuby | |
| app_tarai | 7.69 | 2.14 | 6.80 | 8.40 | YARV | JRuby | MRI | |
| app_uri | 4.70 | 2.58 | 5.10 | error | YARV | MRI | JRuby | |
| io_file_create | 3.17 | 3.56 | 29.82 | error | MRI | YARV | JRuby | |
| io_file_read | 1.22 | 1.58 | 2.30 | error | MRI | YARV | JRuby | |
| io_file_write | 0.68 | 0.52 | 1.25 | error | YARV | MRI | JRuby | |
| interpreter_launch | 0.02 | 0.03 | 1.78 | 0.09 | MRI | YARV | Rubinius | |
| loop_generator | 26.53 | 1.79 | t/o | error | YARV | MRI | None | |
| loop_times | 6.06 | 4.44 | 11.96 | 18.64 | YARV | MRI | JRuby | |
| loop_whileloop | 10.33 | 1.87 | 13.51 | 3.61 | YARV | Rubinius | MRI | |
| loop_whileloop2 | 2.09 | 0.37 | 2.75 | 0.75 | YARV | Rubinius | MRI | |
| so_ackermann | error | 2.02 | 39.58 | 3.70 | YARV | Rubinius | JRuby | |
| so_array | 7.37 | 4.41 | 24.97 | 101.48 | YARV | MRI | JRuby | |
| so_binary_trees | 3.32 | 1.81 | 5.35 | error | YARV | MRI | JRuby | |
| so_concatenate | 2.17 | 1.68 | 5.61 | 18.09 | YARV | MRI | JRuby | |
| so_count_words | 3.96 | 4.24 | 2.45 | error | JRuby | MRI | YARV | |
| so_exception | 4.88 | 4.58 | 7.65 | 145.62 | YARV | MRI | JRuby | |
| so_fannkuch | 82.43 | 60.60 | 62.79 | error | YARV | JRuby | MRI | |
| so_fasta | 11.06 | 8.03 | 22.92 | t/o | YARV | MRI | JRuby | |
| so_k_nucleotide | 6.49 | 6.07 | 9.61 | error | YARV | MRI | JRuby | |
| so_lists | 1.30 | 1.01 | 3.55 | error | YARV | MRI | JRuby | |
| so_mandelbrot | 32.75 | 19.02 | 29.21 | t/o | YARV | JRuby | MRI | |
| so_matrix | 2.54 | 1.14 | 3.29 | 12.31 | YARV | MRI | JRuby | |
| so_meteor_contest | 38.06 | 24.97 | 42.27 | error | YARV | MRI | JRuby | |
| so_nbody | 23.77 | 12.65 | 24.76 | 122.08 | YARV | MRI | JRuby | |
| so_nested_loop | 6.14 | 3.92 | 12.46 | 9.95 | YARV | MRI | Rubinius | |
| so_nsieve | 22.55 | 8.74 | 26.73 | error | YARV | MRI | JRuby | |
| so_nsieve_bits | 52.98 | 10.77 | 67.98 | 89.70 | YARV | MRI | JRuby | |
| so_object | 8.06 | 2.62 | 6.35 | 7.30 | YARV | JRuby | Rubinius | |
| so_partial_sums | 26.52 | 13.35 | 29.71 | 160.15 | YARV | MRI | JRuby | |
| so_pidigits | 6.36 | 7.00 | 11.16 | 8.42 | MRI | YARV | Rubinius | |
| so_random | 2.21 | 0.75 | 2.85 | 105.38 | YARV | MRI | JRuby | |
| so_reverse_complement | 4.36 | 4.33 | error | error | YARV | MRI | None | |
| so_sieve | 0.70 | 0.21 | 1.00 | 3.39 | YARV | MRI | JRuby | |
| so_spectralnorm | 28.46 | 14.42 | 36.27 | 293.51 | YARV | MRI | JRuby | |
| vm1_block | 14.37 | 4.91 | 23.86 | 21.86 | YARV | MRI | Rubinius | |
| vm1_const | 7.02 | 0.96 | 3.15 | 20.11 | YARV | JRuby | MRI | |
| vm1_ensure | 7.42 | 0.04 | 6.25 | 0.93 | YARV | Rubinius | JRuby | |
| vm1_ivar | 6.73 | 4.19 | 2.40 | 5.23 | JRuby | YARV | Rubinius | |
| vm1_ivar_set | 7.43 | 4.42 | 7.30 | 4.85 | YARV | Rubinius | JRuby | |
| vm1_length | 9.73 | 3.23 | 6.65 | 12.75 | YARV | JRuby | MRI | |
| vm1_rescue | 2.87 | 0.24 | 10.55 | 1.11 | YARV | Rubinius | MRI | |
| vm1_simplereturn | 10.84 | 3.12 | 6.35 | 6.08 | YARV | Rubinius | JRuby | |
| vm1_swap | 15.68 | 0.75 | 5.41 | 2.14 | YARV | Rubinius | JRuby | |
| vm2_array | 4.26 | 3.14 | 2.00 | 6.77 | JRuby | YARV | MRI | |
| vm2_case | 2.88 | 0.42 | 4.11 | 3.24 | YARV | MRI | Rubinius | |
| vm2_eval | 14.15 | 60.23 | 111.10 | t/o | MRI | YARV | JRuby | |
| vm2_method | 10.79 | 5.18 | 12.66 | 10.04 | YARV | Rubinius | MRI | |
| vm2_mutex | 3.37 | 3.75 | 7.45 | error | MRI | YARV | JRuby | |
| vm2_poly_method | 14.48 | 6.23 | 31.96 | 13.09 | YARV | Rubinius | MRI | |
| vm2_poly_method_ov | 2.53 | 0.51 | 2.20 | 2.82 | YARV | JRuby | MRI | |
| vm2_proc | 5.42 | 1.96 | 5.66 | 7.04 | YARV | MRI | JRuby | |
| vm2_regexp | 2.71 | 3.52 | 7.36 | 28.52 | MRI | YARV | JRuby | |
| vm2_send | 2.56 | 0.76 | 3.06 | 2.23 | YARV | Rubinius | MRI | |
| vm2_super | 3.04 | 1.26 | 3.15 | 2.38 | YARV | Rubinius | MRI | |
| vm2_unif1 | 1.88 | 0.68 | 2.30 | 1.57 | YARV | Rubinius | MRI | |
| vm2_zsuper | 3.57 | 1.31 | 5.26 | 2.55 | YARV | Rubinius | MRI | |
| vm3_thread_create_join | 0.03 | 0.12 | 0.40 | error | MRI | YARV | JRuby | |
| vm3_thread_mutex | 80.99 | 32.18 | 25.87 | error | JRuby | YARV | MRI | |
| lib_coderay | 0.82 | 0.74 | t/o | error | YARV | MRI | None | |
| lib_json_pure | 0.36 | 0.28 | t/o | error | YARV | MRI | None | |
| interpreter_require | 0.06 | 0.11 | 2.88 | error | MRI | YARV | JRuby |
* means that results of loop_whileloop/loop_whileloop2 are subtracted.
The time for launch is subtracted from all other benchmarks.
| group | # | yarv | jruby | rubinius |
| Mini apps (app) | 11 | 2.3x | 0.8x | 1.3x |
| Interpreter (interpreter) | 2 | 0.5x | 0.0x | 0.2x |
| I/O (io) | 3 | 1.0x | 0.4x | |
| Libraries (lib) | 2 | 1.2x | ||
| Loops (loop) | 4 | 6.8x | 0.7x | 2.0x |
| Shootout (so) | 24 | 1.9x | 0.8x | 0.3x |
| Core (vm) | 24 | 11.3x | 1.2x | 1.7x |
require 'cgi'
puts <<-HEAD
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" version="-//W3C//DTD XHTML 1.1//EN" xml:lang="de">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<title>Ruby 1.8 vs. 1.9 Benchmarks</title>
<style type="text/css" media="screen">
html { font-family: 'Trebuchet MS', Verdana, Arial, sans-serif; }
td {
vertical-align: middle;
white-space: nowrap;
padding: 2px 0.3em;
text-align: left;
border: 1px solid black; border-width: 2px 0;
height: 50px;
}
table {
border-collapse: collapse;
margin: 0;
}
thead td { border-top-width: 0; }
.time { font-size: 0.9em; }
.center { text-align: center; }
.right { text-align: right; }
.newgroup { border-left: 2px solid black; }
.note { color: red; font-weight: bold; }
.error { color: darkred }
.timeout { color: purple; }
.success { color: green; }
dd { font-style: italic; }
.bar {
margin: 0px; padding: 0px;
white-space: nowrap;
font-size: 0.7em;
height: 12px; margin-top: 1px; padding: 0px 0.2em;
border: 1px solid gray;
text-indent: 0.3em;
text-align: right;
}
.mri { background-color: #faa; }
.yarv { background-color: #fbe; }
.jruby { background-color: #fb8; }
.rubinius { background-color: #dba; }
.bar.error { border: none; text-indent: 0px; color: black; }
.xfaster { border-color: green; }
.xslower { border-color: #400; }
</style>
</head>
<body>
<h1>Benchmarks: Ruby 1.8 and Ruby 1.9</h1>
<p>using:</p>
<dl>
<dt>Ruby 1.8 (MRI)</dt>
<dd>ruby 1.8.6 (2007-09-24) [i686-darwin8.10.1]</dd>
<dt>Ruby 1.9 (YARV)</dt>
<dd>ruby 1.9.0 (2007-11-09) [i686-darwin8.10.1]</dd>
<dt>JRuby</dt>
<dd>ruby 1.8.5 (2007-11-10 rev 4842) [i386-jruby1.1b1]</dd>
<dt>Rubinius</dt>
<dd>rubinius 0.8.0 (ruby 1.8.6 compatible) (c1b9f74f8) (11/10/2007) [i686-apple-darwin8.10.1]</dd>
<dt>System</dt>
<dd>Mac OS X, MacBook, Core Duo 1.83GHz, 2GB RAM</dd>
</dl>
<p class="note">
Please note that these are mostly synthetic benchmarks!
</p>
<table>
<thead>
<tr>
<td class="benchmark">Benchmark</td>
<td class="time right" style="width: 4em">MRI</td>
<td class="time right" style="width: 4em">YARV</td>
<td class="time right" style="width: 4em">JRuby</td>
<td class="time right" style="width: 4em">Rubinius</td>
<td class="time right">1st</td>
<td class="time right">2nd</td>
<td class="time right">3rd</td>
<td class="time">(in seconds)</td>
</tr>
</thead>
<tbody>
HEAD
WIDTH_FACTOR = 250
MAX_WIDTH = 600
EXPLAIN = {
'app' => 'Mini apps',
'so' => 'Shootout',
'lib' => 'Libraries',
'vm' => 'Core',
'loop' => 'Loops',
'io' => 'I/O',
'other' => 'Other',
'interpreter' => 'Interpreter',
}
speedups_for = Hash.new do |h, k|
h[k] = { 'yarv' => [], 'jruby' => [], 'rubinius' => [] }
end
looptime = Hash.new([0, 0])
def benchmark_result result
case result
when '-ERROR-'
:error
when '-TIMEOUT-'
:timeout
else
result.to_f
end
end
DATA.read.scan(/^\s*(\w+)\*?\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+) *(.+)?/) do |name, ruby18, ruby19, jruby, rubinius, comment|
next if name == 'Benchmark' # skip header
if false #broken
puts "<tr class=\"broken\">"
puts "<td class=\"benchmark\">%s</td>" % [name] + "<td></td>" * 2 + "<td>[#{$4}]</td>"
puts "</tr>"
next
end
results = [ruby18, ruby19, jruby, rubinius]
results.map! do |result|
benchmark_result result
end
# build HTML table
puts "<tr>"
puts '<td class="benchmark time">%s</td>' % name
newgroup = 'newgroup'
for interpreter, result in %w(MRI YARV JRuby Rubinius).zip results
print '<td class="time right %s %s">' % [interpreter.downcase, newgroup]
case result
when Float
print '<span class="success">%0.2f</span>' % result
when :error
print '<span class="error">error</span>'
when :timeout
print '<span class="timeout">t/o</span>'
end
puts '</td>'
newgroup = false
end
newgroup = 'newgroup'
for place in results.map { |r| r.is_a?(Numeric) ? r : 1.0/0.0 }.sort[0,3]
winner = %w(MRI YARV JRuby Rubinius None)[results.index(place) || 4]
puts '<td class="center time %s %s">%s</td>' % [winner.downcase, newgroup, winner || '-']
newgroup = false
end
puts '<td class="newgroup">'
reference = results.first
unless reference.is_a? Float
reference = results[1]
mri_broken = true
end
if reference.is_a? Float
puts '<div class="bar mri" style="width: %dpx">mri %0.2f seconds (100%%)</div>' % [
WIDTH_FACTOR,
reference
]
results[1..-1].each_with_index do |result, index|
if result.is_a? Float
relative_time = result / reference
interpreter = %w(yarv jruby rubinius)[index]
puts '<div class="bar %s %s" style="width: %dpx">%s %0.1fx (%+0.0f%%%s)</div>' % [
interpreter,
(result <= reference ? 'faster' : 'slower'),
[MAX_WIDTH, WIDTH_FACTOR * relative_time].min,
interpreter,
(reference / result),
100 * (relative_time - 1),
(', cut' if WIDTH_FACTOR * relative_time > MAX_WIDTH),
]
speedup = reference / result
if name[/^(app|loop|so|vm|lib|io|interpreter)/]
speedups_for[$1][interpreter] << speedup
else
speedups_for['other'][interpreter] << speedup
end
else
puts '<div class="bar error" style="width: %dpx">%s: %s</div>' % [
0,
%w(yarv jruby rubinius)[index],
result.to_s,
]
end
end
end
puts '</td>'
puts "</tr>"
end
puts "</tbody></table>"
puts "<p><small>* means that results of loop_whileloop/loop_whileloop2 are subtracted.</small><br />"
puts "<small>The time for launch is subtracted from all other benchmarks.</small></p>"
if true
puts <<-HTML
<table>
<thead>
<tr>
<td class="benchmark">group</td>
<td class="right">#</td>
<td class="right">yarv</td>
<td class="right">jruby</td>
<td class="right">rubinius</td>
</tr>
</thead>
<tbody>
HTML
speedups_for.sort.each do |tag, speedups_by_interpreter|
puts '<tr>'
size = speedups_by_interpreter['yarv'].size
puts "<td>#{EXPLAIN[tag]} (#{tag})</td><td class=\"right\">%d</td>" % [size]
speedups_by_interpreter.
sort_by { |interpreter, speedups| %(yarv jruby rubinius).index interpreter }.
map do |interpreter, speedups|
print '<td class="right">'
unless speedups.empty?
average_speedup = speedups.inject { |sum,x| sum + x } / speedups.size
print '%0.1fx' % average_speedup
end
puts '</td>'
end
puts '</tr>'
end
puts "</tbody></table>"
end
puts <<-BODY
<h3>Sourcecode</h3>
<pre>#{DATA.rewind; CGI.escapeHTML(DATA.read)}</pre>
</body>
</html>
BODY
__END__
Benchmark MRI 1.8.6 YARV 1.9.0 JRuby 1.1 Rubinius 0.8.0
app_answer 0.610 0.140 1.001 0.223
app_erb 2.180 2.101 7.305 -ERROR-
app_factorial -ERROR- 1.090 4.553 1.514
app_fib 7.929 1.959 7.603 3.718
app_mandelbrot 2.757 1.229 3.505 21.347
app_pentomino 114.433 59.071 181.930 -TIMEOUT-
app_raise 2.846 3.425 2.651 3.242
app_strconcat 1.897 1.419 1.801 1.805
app_tak 9.569 2.641 8.553 4.799
app_tarai 7.692 2.140 6.802 8.399
app_uri 4.697 2.581 5.104 -ERROR-
io_file_create 3.169 3.561 29.819 -ERROR-
io_file_read 1.221 1.577 2.300 -ERROR-
io_file_write 0.683 0.521 1.253 -ERROR-
interpreter_launch 0.017 0.033 1.781 0.086
loop_generator 26.527 1.790 -TIMEOUT- -ERROR-
loop_times 6.062 4.436 11.956 18.640
loop_whileloop 10.334 1.869 13.505 3.608
loop_whileloop2 2.086 0.367 2.751 0.746
so_ackermann -ERROR- 2.021 39.582 3.699
so_array 7.368 4.414 24.971 101.479
so_binary_trees 3.318 1.807 5.354 -ERROR-
so_concatenate 2.167 1.680 5.605 18.086
so_count_words 3.955 4.238 2.451 -ERROR-
so_exception 4.880 4.578 7.652 145.619
so_fannkuch 82.430 60.596 62.791 -ERROR-
so_fasta 11.056 8.028 22.916 -TIMEOUT-
so_k_nucleotide 6.492 6.074 9.610 -ERROR-
so_lists 1.300 1.008 3.553 -ERROR-
so_mandelbrot 32.747 19.023 29.210 -TIMEOUT-
so_matrix 2.539 1.140 3.293 12.312
so_meteor_contest 38.065 24.966 42.271 -ERROR-
so_nbody 23.775 12.648 24.761 122.077
so_nested_loop 6.137 3.917 12.458 9.949
so_nsieve 22.552 8.744 26.730 -ERROR-
so_nsieve_bits 52.979 10.770 67.980 89.700
so_object 8.056 2.623 6.353 7.304
so_partial_sums 26.516 13.350 29.715 160.146
so_pidigits 6.363 7.000 11.157 8.417
so_random 2.215 0.749 2.851 105.381
so_reverse_complement 4.360 4.327 -ERROR- -ERROR-
so_sieve 0.702 0.207 1.004 3.386
so_spectralnorm 28.459 14.420 36.268 293.515
vm1_block* 14.372 4.910 23.861 21.857
vm1_const* 7.021 0.962 3.152 20.115
vm1_ensure* 7.422 0.041 6.254 0.932
vm1_ivar* 6.726 4.188 2.402 5.227
vm1_ivar_set* 7.434 4.420 7.304 4.849
vm1_length* 9.735 3.227 6.652 12.748
vm1_rescue* 2.865 0.241 10.555 1.108
vm1_simplereturn* 10.837 3.116 6.354 6.085
vm1_swap* 15.678 0.751 5.407 2.136
vm2_array* 4.263 3.138 2.001 6.773
vm2_case* 2.882 0.421 4.105 3.242
vm2_eval* 14.153 60.226 111.099 -TIMEOUT-
vm2_method* 10.791 5.180 12.657 10.036
vm2_mutex* 3.370 3.751 7.454 -ERROR-
vm2_poly_method* 14.479 6.234 31.964 13.089
vm2_poly_method_ov* 2.527 0.511 2.203 2.823
vm2_proc* 5.422 1.963 5.655 7.038
vm2_regexp* 2.712 3.516 7.356 28.525
vm2_send* 2.560 0.761 3.055 2.230
vm2_super* 3.045 1.257 3.153 2.383
vm2_unif1* 1.875 0.681 2.303 1.574
vm2_zsuper* 3.566 1.312 5.256 2.552
vm3_thread_create_join 0.028 0.119 0.400 -ERROR-
vm3_thread_mutex 80.988 32.180 25.872 -ERROR-
lib_coderay 0.817 0.745 -TIMEOUT- -ERROR-
lib_json_pure 0.362 0.279 -TIMEOUT- -ERROR-
interpreter_require 0.061 0.107 2.877 -ERROR-