Benchmarking Ruby Refinements

Are refinements slow? I wrote a few benchmarks to find out. The answer was surprising.

If you search Google for Ruby refinements without knowing any of the back story you might come away with the idea that refinements are slow.

As they were originally proposed, refinements would have been slow. They would have made it impossible for the interpreter to optimize things like method lookup.

But the actual implementation of refinements is a bit more limited than the original proposal. So I thought it would be interesting to run a series of benchmarks on refinements as exist in Ruby today.

TL;DR

Refinements aren't slow. Or at least they don't seem to be any slower than "normal" methods.

The dummy load

We're going to be benchmarking method calls. So we'll need a few methods.

Here, I'm creating two versions of a silly little method. One is "normal" and the other is inside of a refinement:

# As our "dummy load" we're going to create shrugs. 
# 1 shrug == "¯\_(ツ)_/¯"
# 2 shrugs == "¯\_(ツ)_/¯¯\_(ツ)_/¯"
# ...etc.

SHRUG = \_(ツ)_/¯"

# We'll make a refinement that generates shrugs
module Shruggable
  refine Fixnum do
    def shrugs
      SHRUG * self
    end
  end
end

# ...and we'll make a normal method that also generates shrugs
def integer_to_shrugs(n)
  SHRUG * n
end

We can't use the refinement directly. It has to be activated via a using statement. So I'll create two classes that behave identically. One uses the refinement, and the other doesn't.

class TestUsing
  using Shruggable
  def noop
  end

  def shrug
    10.shrugs
  end
end

class TestWithoutUsing
  def noop
  end

  def shrug
    integer_to_shrugs(10)
  end
end

The benchmarks

I wanted to know if it was any slower to instantiate objects using refinements, or to call methods added via refinements.

All benchmarks were run with MRI 2.2.2 on OSX El Capitan.

Object creation

Does the "using" keyword make a class slower to initialize? Nope.

Benchmark.ips do |bm|
  bm.report("class initialization") { TestUsing.new }
  bm.report("class initialization WITH using") { TestWithoutUsing.new }
  bm.compare!
end

# Calculating -------------------------------------
# class initialization   142.929k i/100ms
# class initialization WITH using
#                        145.323k i/100ms
# -------------------------------------------------
# class initialization      5.564M (± 8.3%) i/s -     27.728M
# class initialization WITH using
#                           5.619M (± 7.4%) i/s -     28.047M
# Comparison:
# class initialization WITH using:  5618601.3 i/s
# class initialization:  5564116.5 i/s - 1.01x slower

Method calls

Do refinements affect "normal" method lookup speed? Nope.

Benchmark.ips do |bm|
  bm.report("run method") { TestUsing.new.noop }
  bm.report("run method in class WITH using") { TestWithoutUsing.new.noop }
  bm.compare!
end

# Calculating -------------------------------------
#           run method   141.905k i/100ms
# run method in class WITH using
#                        144.435k i/100ms
# -------------------------------------------------
#           run method      5.010M (± 6.4%) i/s -     24.975M
# run method in class WITH using
#                           5.086M (± 5.3%) i/s -     25.421M
# Comparison:
# run method in class WITH using:  5086262.3 i/s
#           run method:  5010273.6 i/s - 1.02x slower

Is using a method from a refinement slower than using an equivalent "normal" method? Nope.

Benchmark.ips do |bm|
  bm.report("shrug") { TestUsing.new.shrug }
  bm.report("shrug via refinement") { TestWithoutUsing.new.shrug }
  bm.compare!
end

# Calculating -------------------------------------
#                shrug    96.089k i/100ms
# shrug via refinement    95.559k i/100ms
# -------------------------------------------------
#                shrug      1.825M (± 9.3%) i/s -      9.128M
# shrug via refinement      1.929M (± 6.2%) i/s -      9.651M

# Comparison:
# shrug via refinement:  1928841.5 i/s
#                shrug:  1825069.4 i/s - 1.06x slower

Stacking the deck

Can I do anything to make a refinement benchmark slower than my control? ¯\_(ツ)_/¯


# Does repeated evaluation of the `using` keyword affect performance. Only slightly. 
# This is an unfair test, but I really wanted to force refinements to be slower
# in SOME use case :)
Benchmark.ips do |bm|
  bm.report("inline shrug") { integer_to_shrugs(10) }
  bm.report("inline shrug via refinement") do
    using Shruggable
    10.shrugs
  end
  bm.compare!
end

# Calculating -------------------------------------
#         inline shrug   100.460k i/100ms
# inline shrug via refinement
#                         72.131k i/100ms
# -------------------------------------------------
#         inline shrug      2.507M (± 5.2%) i/s -     12.557M
# inline shrug via refinement
#                           1.498M (± 4.3%) i/s -      7.502M

# Comparison:
#         inline shrug:  2506663.9 i/s
# inline shrug via refinement:  1497747.6 i/s - 1.67x slower

The full code

If you'd like to run the benchmark yourself, here's the code.

require 'benchmark/ips'


# As our "dummy load" we're going to create shrugs. 
# 1 shrug == "¯\_(ツ)_/¯"
# 2 shrugs == "¯\_(ツ)_/¯¯\_(ツ)_/¯"
# ...etc.

SHRUG = \_(ツ)_/¯"

# We'll make a refinement that generates shrugs
module Shruggable
  refine Fixnum do
    def shrugs
      SHRUG * self
    end
  end
end

# ...and we'll make a normal method that also generates shrugs
def integer_to_shrugs(n)
  SHRUG * n
end

# Now we'll define two classes. The first uses refinments. The second doesn't. 
class TestUsing
  using Shruggable
  def noop
  end

  def shrug
    10.shrugs
  end
end

class TestWithoutUsing
  def noop
  end

  def shrug
    integer_to_shrugs(10)
  end
end

# Does the "using" keyword make a class slower to initialize? Nope. 
Benchmark.ips do |bm|
  bm.report("class initialization") { TestUsing.new }
  bm.report("class initialization WITH using") { TestWithoutUsing.new }
  bm.compare!
end

# Calculating -------------------------------------
# class initialization   142.929k i/100ms
# class initialization WITH using
#                        145.323k i/100ms
# -------------------------------------------------
# class initialization      5.564M (± 8.3%) i/s -     27.728M
# class initialization WITH using
#                           5.619M (± 7.4%) i/s -     28.047M
# Comparison:
# class initialization WITH using:  5618601.3 i/s
# class initialization:  5564116.5 i/s - 1.01x slower

# Do refinements affect "normal" method lookup speed? Nope. 
Benchmark.ips do |bm|
  bm.report("run method") { TestUsing.new.noop }
  bm.report("run method in class WITH using") { TestWithoutUsing.new.noop }
  bm.compare!
end

# Calculating -------------------------------------
#           run method   141.905k i/100ms
# run method in class WITH using
#                        144.435k i/100ms
# -------------------------------------------------
#           run method      5.010M (± 6.4%) i/s -     24.975M
# run method in class WITH using
#                           5.086M (± 5.3%) i/s -     25.421M
# Comparison:
# run method in class WITH using:  5086262.3 i/s
#           run method:  5010273.6 i/s - 1.02x slower


# Is using a method from a refinement slower than using an equivalent "normal" method? Nope. 
Benchmark.ips do |bm|
  bm.report("shrug") { TestUsing.new.shrug }
  bm.report("shrug via refinement") { TestWithoutUsing.new.shrug }
  bm.compare!
end

# Calculating -------------------------------------
#                shrug    96.089k i/100ms
# shrug via refinement    95.559k i/100ms
# -------------------------------------------------
#                shrug      1.825M (± 9.3%) i/s -      9.128M
# shrug via refinement      1.929M (± 6.2%) i/s -      9.651M

# Comparison:
# shrug via refinement:  1928841.5 i/s
#                shrug:  1825069.4 i/s - 1.06x slower


# Does repeated evaluation of the `using` keyword affect performance. Only slightly. 
# This is an unfair test, but I really wanted to force refinements to be slower
# in SOME use case :)
Benchmark.ips do |bm|
  bm.report("inline shrug") { integer_to_shrugs(10) }
  bm.report("inline shrug via refinement") do
    using Shruggable
    10.shrugs
  end
  bm.compare!
end

# Calculating -------------------------------------
#         inline shrug   100.460k i/100ms
# inline shrug via refinement
#                         72.131k i/100ms
# -------------------------------------------------
#         inline shrug      2.507M (± 5.2%) i/s -     12.557M
# inline shrug via refinement
#                           1.498M (± 4.3%) i/s -      7.502M

# Comparison:
#         inline shrug:  2506663.9 i/s
# inline shrug via refinement:  1497747.6 i/s - 1.67x slower
What to do next:
  1. Sign up for a FREE Honeybadger account
    Honeybadger helps you find and fix errors before your users can even report them. Get set up in minutes and check monitoring off your to-do list.
    Try Honeybadger for FREE
  2. Get the Honeybadger newsletter
    Each month we share news, best practices, and stories from the DevOps & monitoring community—exclusively for developers like you.
    author photo

    Starr Horne

    Starr Horne is a Rubyist and Chief JavaScripter at Honeybadger.io. When she's not neck-deep in other people's bugs, she enjoys making furniture with traditional hand-tools, reading history and brewing beer in her garage in Seattle.

    More articles by Starr Horne
    Stop wasting time manually checking logs for errors!

    Try the only application health monitoring tool that allows you to track application errors, uptime, and cron jobs in one simple platform.

    • Know when critical errors occur, and which customers are affected.
    • Respond instantly when your systems go down.
    • Improve the health of your systems over time.
    • Fix problems before your customers can report them!

    As developers ourselves, we hated wasting time tracking down errors—so we built the system we always wanted.

    Honeybadger tracks everything you need and nothing you don't, creating one simple solution to keep your application running and error free so you can do what you do best—release new code. Try it free and see for yourself.

    Try Honeybadger for FREE
    No credit card needed - Simple 5-minute setup

    Learn more

    "We've looked at a lot of error management systems. Honeybadger is head and shoulders above the rest and somehow gets better with every new release."
    — Michael Smith, Cofounder & CTO of YvesBlue

    Honeybadger is trusted by top companies like:

    “We've looked at a lot of error management systems. Honeybadger is head and shoulders above the rest and somehow gets better with every new release.” 
    Michael Smith
    Try Honeybadger for FREE
    Are you using Sentry, Rollbar, Bugsnag, or Airbrake for your monitoring? Honeybadger includes error tracking with a whole suite of amazing monitoring tools — all for probably less than you're paying now. Discover why so many companies are switching to Honeybadger here.
    Try Honeybadger for FREE
    Stop digging through chat logs to find the bug-fix someone mentioned last month. Honeybadger's built-in issue tracker keeps discussion central to each error, so that if it pops up again you'll be able to pick up right where you left off.
    Try Honeybadger for FREE
    "Wow — Customers are blown away that I email them so quickly after an error."
    Chris Patton
    Try Honeybadger for FREE