Introducing Breadcrumbs

We added a useful debugging tool called Breadcrumbs. Check them out, and perhaps you will never need a random debug log in production ever again.

Have you ever dealt with an error in production, and no matter what you try, you can't replicate the issue on your development or staging environments? Often the next step is to gather more data by tossing a debug log at production. If you don't have a good way to correlate logs with a request it can be frustrating, especially during an incident.

We added a feature to help, and it's called Breadcrumbs.

A Breadcrumb is very much like a log event, but it's stored and reported along side an error. Like a log, a Breadcrumb contains a message, but it can also hold metadata (in the form of a hashmap). A set of breadcrumbs are collected throughout the life of a request (or job invocation), and are immediately dropped unless an error is reported.

Adding your own breadcrumbs to the stack is simple. Just make a call to Honeybadger.add_breadcrumb anywhere in your code:

Honeybadger.add_breadcrumb("Loading User", metadata: {
  user_name: user_name,

And if an error is reported after it, you should see it in the Breadcrumb stack:

Oh user_name looks empty, that might cause problems.

Automatic breadcrumbs

The Honeybadger Ruby library contains hooks into Ruby & Rails to automatically collect breadcrumbs. For example, all log messages emitted in Production (sent through the Logger class) are captured and created as breadcrumbs. We also tie into Rails Instrumentation to gather breadcrumbs for Controller Actions, SQL Queries, Active Job invocations, etc...

How can you use it?

Breadcrumbs are currently available via our Ruby client. You must update to version 4.4.0 and be sure to enable it in the config as it will be disabled by default until we release a 5.0.0 version of the gem, to ensure we work out any kinks.

Please try it out and let us know if you run into any issues.

Extending Breadcrumbs

Now that Breadcrumbs have been formally introduced, let's see a quick example of how to extend them. If you enable Breadcrumbs in a Rails app, you will get some standard instrumentation breadcrumbs attached for free, but what if we want more?

Let's say we want to create a Breadcrumb every time our app sends out an HTTP request. This info might come in handy while debugging.

A simple way to accomplish this would be to call Honeybadger.add_breadcrumb at each request invocation.

def send_a_message
  res ="/message", { user:, body: "Hey!" }.to_json)
  Honeybadger.add_breadcrumb("Request: /message", metadata: { user: })

Here we store a breadcrumb after every POST to /message. The next time our app throws an error after sending this message, we should see which user sent the message and when it happened in relation to the error, Yay!

This is a bit cumbersome though, as we would need an add_breadcrumb call at each location we send out a request. I do want to note, however, there are advantages to creating breadcrumbs like this. You can be very specific about what metadata you want to capture with this method. Often this is a great way to gather targeted information motivated by a bug in production.

Really I just want to know when and where a request goes out. It would also be nice if a library could do most of the work for me ;).

Instrumenting with Faraday

We are going to cheat a little and assume you are using the popular Faraday request library.

We could build our own middleware to accomplish our task, but instead I want to use Rails instrumentation as a broker. Luckily, there is middleware provided by the faraday_middleware gem that will emit events for us. Ensure the gem is in your Gemfile and also make sure the instrumentation middleware is injected into the connection.

connection = do |conn|
  conn.use :instrumentation
  conn.adapter Faraday.default_adapter

This can require a little work if you don't share your connection inside your app. If you are lazy you can also prepend the ConnectionOptions to ensure that all requests have the middleware enabled. Be careful though, as this will instrument any Faraday requests that happen within your app and included gems as well (which you might want)!

Using Rails instrumentation has some nice side effects, one being that we can make multiple subscriptions for other use cases (say for general logging purposes).

Binge watching our requests

Now that our Faraday requests are instrumented, we can subscribe and get to creating some breadcrumbs:

ActiveSupport::Notifications.subscribe "request.faraday" do |_, _, _, _, data|
  method = data[:method].to_s.upcase
  metadata = data.to_h.slice(:url, :status).merge({method: method})
  Honeybadger.add_breadcrumb("#{method}: #{metadata[:url]}", category: "request", metadata: metadata)

Honeybadger is now adding a breadcrumb for each outgoing request!

Notice we are only inspecting a few data points from our requests. We don't add outgoing or response body payloads as there is a good chance that sensitive data could leak into our breadcrumb metadata.

Let us know how it goes!

We hope that Breadcrumbs will be a helpful addition your debugging toolbox. Try it out, and give us a shout if there is anything you would like to see added.

What to do next:
  1. Try Honeybadger for FREE
    Honeybadger helps you find and fix errors before your users can even report them. Get set up in minutes and check monitoring off your to-do list.
    Start free trial
    Easy 5-minute setup — No credit card required
  2. Get the Honeybadger newsletter
    Each month we share news, best practices, and stories from the DevOps & monitoring community—exclusively for developers like you.
    author photo

    Kevin Webster

    Kevin is the freshest Honeybadger (both in time and breakdancing abilities). Kevin has been building things with software since his dad brought home the family IBM 386. He fancies himself a bit of a programming polyglot. When he's not compromising his ability to write Ruby by learning new languages, he enjoys hiking through the Oregon wilderness, hanging with his family, or watching cringeworthy b-list movies.

    More articles by Kevin Webster
    Stop wasting time manually checking logs for errors!

    Try the only application health monitoring tool that allows you to track application errors, uptime, and cron jobs in one simple platform.

    • Know when critical errors occur, and which customers are affected.
    • Respond instantly when your systems go down.
    • Improve the health of your systems over time.
    • Fix problems before your customers can report them!

    As developers ourselves, we hated wasting time tracking down errors—so we built the system we always wanted.

    Honeybadger tracks everything you need and nothing you don't, creating one simple solution to keep your application running and error free so you can do what you do best—release new code. Try it free and see for yourself.

    Start free trial
    Simple 5-minute setup — No credit card required

    Learn more

    "We've looked at a lot of error management systems. Honeybadger is head and shoulders above the rest and somehow gets better with every new release."
    — Michael Smith, Cofounder & CTO of YvesBlue

    Honeybadger is trusted by top companies like:

    “Everyone is in love with Honeybadger ... the UI is spot on.”
    Molly Struve, Sr. Site Reliability Engineer, Netflix
    Start free trial