Benchmarking Ruby Refinements

Are refinements slow? I wrote a few benchmarks to find out. The answer was surprising.

If you search Google for Ruby refinements without knowing any of the back story you might come away with the idea that refinements are slow.

As they were originally proposed, refinements would have been slow. They would have made it impossible for the interpreter to optimize things like method lookup.

But the actual implementation of refinements is a bit more limited than the original proposal. So I thought it would be interesting to run a series of benchmarks on refinements as exist in Ruby today.

TL;DR

Refinements aren't slow. Or at least they don't seem to be any slower than "normal" methods.

The dummy load

We're going to be benchmarking method calls. So we'll need a few methods.

Here, I'm creating two versions of a silly little method. One is "normal" and the other is inside of a refinement:

# As our "dummy load" we're going to create shrugs. 
# 1 shrug == "¯\_(ツ)_/¯"
# 2 shrugs == "¯\_(ツ)_/¯¯\_(ツ)_/¯"
# ...etc.

SHRUG = \_(ツ)_/¯"

# We'll make a refinement that generates shrugs
module Shruggable
  refine Fixnum do
    def shrugs
      SHRUG * self
    end
  end
end

# ...and we'll make a normal method that also generates shrugs
def integer_to_shrugs(n)
  SHRUG * n
end

We can't use the refinement directly. It has to be activated via a using statement. So I'll create two classes that behave identically. One uses the refinement, and the other doesn't.

class TestUsing
  using Shruggable
  def noop
  end

  def shrug
    10.shrugs
  end
end

class TestWithoutUsing
  def noop
  end

  def shrug
    integer_to_shrugs(10)
  end
end

The benchmarks

I wanted to know if it was any slower to instantiate objects using refinements, or to call methods added via refinements.

All benchmarks were run with MRI 2.2.2 on OSX El Capitan.

Object creation

Does the "using" keyword make a class slower to initialize? Nope.

Benchmark.ips do |bm|
  bm.report("class initialization") { TestUsing.new }
  bm.report("class initialization WITH using") { TestWithoutUsing.new }
  bm.compare!
end

# Calculating -------------------------------------
# class initialization   142.929k i/100ms
# class initialization WITH using
#                        145.323k i/100ms
# -------------------------------------------------
# class initialization      5.564M (± 8.3%) i/s -     27.728M
# class initialization WITH using
#                           5.619M (± 7.4%) i/s -     28.047M
# Comparison:
# class initialization WITH using:  5618601.3 i/s
# class initialization:  5564116.5 i/s - 1.01x slower

Method calls

Do refinements affect "normal" method lookup speed? Nope.

Benchmark.ips do |bm|
  bm.report("run method") { TestUsing.new.noop }
  bm.report("run method in class WITH using") { TestWithoutUsing.new.noop }
  bm.compare!
end

# Calculating -------------------------------------
#           run method   141.905k i/100ms
# run method in class WITH using
#                        144.435k i/100ms
# -------------------------------------------------
#           run method      5.010M (± 6.4%) i/s -     24.975M
# run method in class WITH using
#                           5.086M (± 5.3%) i/s -     25.421M
# Comparison:
# run method in class WITH using:  5086262.3 i/s
#           run method:  5010273.6 i/s - 1.02x slower

Is using a method from a refinement slower than using an equivalent "normal" method? Nope.

Benchmark.ips do |bm|
  bm.report("shrug") { TestUsing.new.shrug }
  bm.report("shrug via refinement") { TestWithoutUsing.new.shrug }
  bm.compare!
end

# Calculating -------------------------------------
#                shrug    96.089k i/100ms
# shrug via refinement    95.559k i/100ms
# -------------------------------------------------
#                shrug      1.825M (± 9.3%) i/s -      9.128M
# shrug via refinement      1.929M (± 6.2%) i/s -      9.651M

# Comparison:
# shrug via refinement:  1928841.5 i/s
#                shrug:  1825069.4 i/s - 1.06x slower

Stacking the deck

Can I do anything to make a refinement benchmark slower than my control? ¯\_(ツ)_/¯


# Does repeated evaluation of the `using` keyword affect performance. Only slightly. 
# This is an unfair test, but I really wanted to force refinements to be slower
# in SOME use case :)
Benchmark.ips do |bm|
  bm.report("inline shrug") { integer_to_shrugs(10) }
  bm.report("inline shrug via refinement") do
    using Shruggable
    10.shrugs
  end
  bm.compare!
end

# Calculating -------------------------------------
#         inline shrug   100.460k i/100ms
# inline shrug via refinement
#                         72.131k i/100ms
# -------------------------------------------------
#         inline shrug      2.507M (± 5.2%) i/s -     12.557M
# inline shrug via refinement
#                           1.498M (± 4.3%) i/s -      7.502M

# Comparison:
#         inline shrug:  2506663.9 i/s
# inline shrug via refinement:  1497747.6 i/s - 1.67x slower

The full code

If you'd like to run the benchmark yourself, here's the code.

require 'benchmark/ips'


# As our "dummy load" we're going to create shrugs. 
# 1 shrug == "¯\_(ツ)_/¯"
# 2 shrugs == "¯\_(ツ)_/¯¯\_(ツ)_/¯"
# ...etc.

SHRUG = \_(ツ)_/¯"

# We'll make a refinement that generates shrugs
module Shruggable
  refine Fixnum do
    def shrugs
      SHRUG * self
    end
  end
end

# ...and we'll make a normal method that also generates shrugs
def integer_to_shrugs(n)
  SHRUG * n
end

# Now we'll define two classes. The first uses refinments. The second doesn't. 
class TestUsing
  using Shruggable
  def noop
  end

  def shrug
    10.shrugs
  end
end

class TestWithoutUsing
  def noop
  end

  def shrug
    integer_to_shrugs(10)
  end
end

# Does the "using" keyword make a class slower to initialize? Nope. 
Benchmark.ips do |bm|
  bm.report("class initialization") { TestUsing.new }
  bm.report("class initialization WITH using") { TestWithoutUsing.new }
  bm.compare!
end

# Calculating -------------------------------------
# class initialization   142.929k i/100ms
# class initialization WITH using
#                        145.323k i/100ms
# -------------------------------------------------
# class initialization      5.564M (± 8.3%) i/s -     27.728M
# class initialization WITH using
#                           5.619M (± 7.4%) i/s -     28.047M
# Comparison:
# class initialization WITH using:  5618601.3 i/s
# class initialization:  5564116.5 i/s - 1.01x slower

# Do refinements affect "normal" method lookup speed? Nope. 
Benchmark.ips do |bm|
  bm.report("run method") { TestUsing.new.noop }
  bm.report("run method in class WITH using") { TestWithoutUsing.new.noop }
  bm.compare!
end

# Calculating -------------------------------------
#           run method   141.905k i/100ms
# run method in class WITH using
#                        144.435k i/100ms
# -------------------------------------------------
#           run method      5.010M (± 6.4%) i/s -     24.975M
# run method in class WITH using
#                           5.086M (± 5.3%) i/s -     25.421M
# Comparison:
# run method in class WITH using:  5086262.3 i/s
#           run method:  5010273.6 i/s - 1.02x slower


# Is using a method from a refinement slower than using an equivalent "normal" method? Nope. 
Benchmark.ips do |bm|
  bm.report("shrug") { TestUsing.new.shrug }
  bm.report("shrug via refinement") { TestWithoutUsing.new.shrug }
  bm.compare!
end

# Calculating -------------------------------------
#                shrug    96.089k i/100ms
# shrug via refinement    95.559k i/100ms
# -------------------------------------------------
#                shrug      1.825M (± 9.3%) i/s -      9.128M
# shrug via refinement      1.929M (± 6.2%) i/s -      9.651M

# Comparison:
# shrug via refinement:  1928841.5 i/s
#                shrug:  1825069.4 i/s - 1.06x slower


# Does repeated evaluation of the `using` keyword affect performance. Only slightly. 
# This is an unfair test, but I really wanted to force refinements to be slower
# in SOME use case :)
Benchmark.ips do |bm|
  bm.report("inline shrug") { integer_to_shrugs(10) }
  bm.report("inline shrug via refinement") do
    using Shruggable
    10.shrugs
  end
  bm.compare!
end

# Calculating -------------------------------------
#         inline shrug   100.460k i/100ms
# inline shrug via refinement
#                         72.131k i/100ms
# -------------------------------------------------
#         inline shrug      2.507M (± 5.2%) i/s -     12.557M
# inline shrug via refinement
#                           1.498M (± 4.3%) i/s -      7.502M

# Comparison:
#         inline shrug:  2506663.9 i/s
# inline shrug via refinement:  1497747.6 i/s - 1.67x slower
author photo

Starr Horne

Starr Horne is a Rubyist and Chief Javascripter at Honeybadger.io. When he's not neck-deep in other people's bugs, he enjoys making furniture with traditional hand-tools, reading history and brewing beer in his garage in Seattle.

“We’ve looked at a lot of error management systems. Honeybadger is head and shoulders above the rest and somehow gets better with every new release.”
Michael Smith
Try Error Monitoring Free for 15 Days
Are you using Bugsnag, Rollbar, or Airbrake for your monitoring? Honeybadger includes exception, uptime, and check-in monitoring — all for probably less than you’re paying now. Discover why so many companies are switching to Honeybadger here.
Try Error Monitoring Free for 15 Days
Stop digging through chat logs to find the bug-fix someone mentioned last month. Honeybadger's built-in issue tracker keeps discussion central to each error, so that if it pops up again you'll be able to pick up right where you left off.
Try Error Monitoring Free for 15 Days
"Wow — Customers are blown away that I email them so quickly after an error."
Chris Patton
Try Error Monitoring Free for 15 Days