A couple people posted responses to the benchmarks on my Sun blog, so here they are.
From Susan Potter:
Bryan,
I wrote a benchmarking blog post in October last year relating to the difference in code-level optimizations between Ruby 1.8.6 and Ruby 1.9.0 and how the roles were reversed in these two versions: (link)
And her direct response:
This is a response to Bryan Donovan’s blog post called ways to pass options.
Bryan’s original benchmark was run on Ruby 1.8.4. I will be running the benchmarks on Ruby 1.8.6 and Ruby 1.9 to demonstrate performance reversal of code level optimizations between Ruby versions (as I did in Ruby performance reversal benchmark 1.8 to 1.9).
I will use Bryan’s Ruby benchmark code to run for both Ruby 1.8.6 and Ruby 1.9.0 (official).Ruby 1.8.6 Results
Below are the results of running the options passing benchmark in Ruby 1.8.6:
With no option values passed Rehearsal ------------------------------------------- delete: 0.290000 0.050000 0.340000 ( 0.428564) merge: 0.700000 0.190000 0.890000 ( 1.017415) or_nil: 0.370000 0.050000 0.420000 ( 0.509209) ---------------------------------- total: 1.650000sec user system total real delete: 0.300000 0.050000 0.350000 ( 0.428401) merge: 0.760000 0.130000 0.890000 ( 1.010474) or_nil: 0.360000 0.050000 0.410000 ( 0.490170) With some option values passed Rehearsal ------------------------------------------- delete: 0.360000 0.060000 0.420000 ( 0.505094) merge: 0.890000 0.130000 1.020000 ( 1.150165) or_nil: 0.390000 0.070000 0.460000 ( 0.563713) ---------------------------------- total: 1.900000sec user system total real delete: 0.340000 0.060000 0.400000 ( 0.489191) merge: 0.920000 0.110000 1.030000 ( 1.149708) or_nil: 0.350000 0.090000 0.440000 ( 0.528287) With all option values passed Rehearsal ------------------------------------------- delete: 0.420000 0.050000 0.470000 ( 0.547446) merge: 0.970000 0.130000 1.100000 ( 1.228093) or_nil: 0.420000 0.050000 0.470000 ( 0.574593) ---------------------------------- total: 2.040000sec user system total real delete: 0.400000 0.050000 0.450000 ( 0.485888) merge: 0.960000 0.140000 1.100000 ( 1.237509) or_nil: 0.400000 0.060000 0.460000 ( 0.536063)
Ruby 1.9.0 Results
Below are the results of running the options passing benchmark in Ruby 1.9.0:
With no option values passed Rehearsal ------------------------------------------- delete: 0.150000 0.000000 0.150000 ( 0.230391) merge: 0.540000 0.010000 0.550000 ( 0.645735) or_nil: 0.120000 0.000000 0.120000 ( 0.188468) ---------------------------------- total: 0.820000sec user system total real delete: 0.160000 0.000000 0.160000 ( 0.238828) merge: 0.520000 0.000000 0.520000 ( 0.610940) or_nil: 0.120000 0.000000 0.120000 ( 0.180744) With some option values passed Rehearsal ------------------------------------------- delete: 0.220000 0.000000 0.220000 ( 0.294110) merge: 0.690000 0.010000 0.700000 ( 0.803382) or_nil: 0.220000 0.000000 0.220000 ( 0.297869) ---------------------------------- total: 1.140000sec user system total real delete: 0.210000 0.000000 0.210000 ( 0.283983) merge: 0.700000 0.000000 0.700000 ( 0.802766) or_nil: 0.230000 0.000000 0.230000 ( 0.295433) With all option values passed Rehearsal ------------------------------------------- delete: 0.270000 0.010000 0.280000 ( 0.348052) merge: 0.770000 0.040000 0.810000 ( 0.947252) or_nil: 0.240000 0.010000 0.250000 ( 0.336790) ---------------------------------- total: 1.340000sec user system total real delete: 0.280000 0.000000 0.280000 ( 0.660810) merge: 0.750000 0.000000 0.750000 ( 1.618925) or_nil: 0.240000 0.000000 0.240000 ( 0.478159)
In Ruby 1.8.6 the delete code is consistently more efficient, whereas in Ruby 1.9.0 the or_nil code is mostly more efficient, except for the With some option values passed scenario.
This once again shows that when optimizing performance on the code level, you need to be careful that you justify your micro-optimizations before you create code spaghetti just “because….”. Readability and maintainability is most important and when needed, only then should you optimize.
Charles Oliver Nutter’s Response
Charles Oliver Nutter (of JRuby) responded as well:
Stumbled on your blog post and ran the same numbers in JRuby on soylatte Java 6 on basically the same machine. My numbers for Ruby 1.8.6p111 were roughly the same as yours.
Here’s Ruby:
user system total real delete: 0.200000 0.000000 0.200000 ( 0.207049) merge: 0.710000 0.010000 0.720000 ( 0.720514) or_nil: 0.260000 0.000000 0.260000 ( 0.256742) With some option values passed user system total real delete: 0.250000 0.000000 0.250000 ( 0.260502) merge: 0.810000 0.010000 0.820000 ( 0.818434) or_nil: 0.290000 0.000000 0.290000 ( 0.294220) With all option values passed user system total real delete: 0.320000 0.000000 0.320000 ( 0.319335) merge: 0.890000 0.010000 0.900000 ( 0.896330) or_nil: 0.330000 0.000000 0.330000 ( 0.336673)
And here’s JRuby:
With no option values passed user system total real delete: 0.133000 0.000000 0.133000 ( 0.133000) merge: 0.318000 0.000000 0.318000 ( 0.319000) or_nil: 0.474000 0.000000 0.474000 ( 0.474000) With some option values passed user system total real delete: 0.170000 0.000000 0.170000 ( 0.169000) merge: 0.295000 0.000000 0.295000 ( 0.295000) or_nil: 0.145000 0.000000 0.145000 ( 0.146000) With all option values passed user system total real delete: 0.154000 0.000000 0.154000 ( 0.154000) merge: 0.289000 0.000000 0.289000 ( 0.288000) or_nil: 0.101000 0.000000 0.101000 ( 0.100000)
Looks like something’s artificially slowing down the “or_nil” case in the first scenario, but the rest of the numbers look pretty solid.
Interesting results all around. I’m not going to lose sleep over the milliseconds I’m saving/losing with various approaches, so I’ll likely use the ops = ops[:foo] || ‘default’ approach for readability.