Sometimes benchmark results are skewed because code executed earlier encounters different garbage collection overheads than that run later. #bmbm attempts to minimize this effect by running the tests twice, the first time as a rehearsal in order to get the runtime environment stable, the second time for real. GC.start is executed before the start of each of the real timings; the cost of this is not included in the timings. In reality, though, there’s only so much that #bmbm can do, and the results are not guaranteed to be isolated from garbage collection and other effects.

Because #bmbm takes two passes through the tests, it can calculate the required label width.

require 'benchmark'

array = (1..1000000).map { rand }

Benchmark.bmbm do |x|
  x.report("sort!") { array.dup.sort! }
  x.report("sort")  { array.dup.sort  }
end

Generates:

Rehearsal -----------------------------------------
sort!  11.928000   0.010000  11.938000 ( 12.756000)
sort   13.048000   0.020000  13.068000 ( 13.857000)
------------------------------- total: 25.006000sec

            user     system      total        real
sort!  12.959000   0.010000  12.969000 ( 13.793000)
sort   12.007000   0.000000  12.007000 ( 12.791000)

#bmbm yields a Benchmark::Job object and returns an array of Benchmark::Tms objects.

Source Code

# File benchmark.rb, line 248
def bmbm(width = 0, &blk) # :yield: job
  job = Job.new(width)
  yield(job)
  width = job.width
  sync = STDOUT.sync
  STDOUT.sync = true

  # rehearsal
  print "Rehearsal "
  puts '-'*(width+CAPTION.length - "Rehearsal ".length)
  list = []
  job.list.each{|label,item|
    print(label.ljust(width))
    res = Benchmark::measure(&item)
    print res.format()
    list.push res
  }
  sum = Tms.new; list.each{|i| sum += i}
  ets = sum.format("total: %tsec")
  printf("%s %s\n\n",
         "-"*(width+CAPTION.length-ets.length-1), ets)

  # take
  print ' '*width, CAPTION
  list = []
  ary = []
  job.list.each{|label,item|
    GC::start
    print label.ljust(width)
    res = Benchmark::measure(&item)
    print res.format()
    ary.push res
    list.push [label, res]
  }

  STDOUT.sync = sync
  ary
end