Cross-cluster Associations in Rails 7

One of the beauties of the Rails framework is the ability to utilize associations in your models. These associations allow you to access collections of records in your code with pleasant syntax, abstracting away the need to write underlying SQL queries. For example, let's consider this basic relationship of objects:

# table doctors in database professionals 
class Doctor < ProfessionalsRecord 
  has_many :appointments, through: :patients 
  has_many :patients 
end 

# table patients in database patients 
class Patient < PatientsRecord
  has_many :appointments 
  has_many :doctors 
end 

# table appointments in database default 
class Appointment < ApplicationRecord 
  has_many :doctors, through: :patients 
  has_many :patients 
end 

As you can imagine, there might be a page in our application that shows a list of appointments for a doctor’s office. To accomplish this, in the backend, we could simply write doctor.appointments. This would be fine if all of these relations were in the same database. Rails loads associations lazily, which means that it executes the join when the data is actually requested. In this case, the SQL that would be performed is as follows:

SELECT appointments.* FROM appointments INNER JOIN patients ON appointments.patient_id = patient.id WHERE patients.doctor_id = ?

However, if we tried to run this with the data across multiple databases, Rails would throw an ActiveRecord::StatementInvalid error. While workarounds exist, they're typically less performant and require a lot of extra work to write custom SQL queries that are error prone and more difficult to maintain.

Why Multiple Databases?

Before we go further into how to utilize cross-cluster associations, let's briefly discuss why you might be utilizing multiple databases with Rails. One of the most common reasons is you want to have redundancy in your infrastructure by having a database replica. Another reason is as your application scales, you may want to segregate your database into multiple smaller databases that each have their own responsibility (e.g., one for users/accounts and one for logging). This allows you to potentially store more critical data on a high-availability database cluster while keeping the less critical data on a cheaper configuration. Finally, if you are migrating from an old schema to a new one, it is easier/safer to setup a new database for this.

While ActiveRecord has supported multiple databases prior to Rails 6, it was with the Rails 6 upgrade that users were able to manage them. You can read more about that here.

How to Utilize Cross-Cluster Associations

Let's go back to our original error from the first section. With Rails 7, there is an easy way to solve this issue! All you have to do is add disable_joins: true in your model associations. Going back to our original example, we would have the following:

class Doctor < ProfessionalsRecord 
  has_many :appointments, through: :patients, disable_joins: true 
  has_many :patients 
end 

What exactly is going on here? This tells Rails Active Record not to attempt the join lazily, and instead, it will perform two separate queries to fetch the records:

SELECT "patients"."id" FROM "patients" WHERE "patients"."doctor_id" = ?  [["doctor_id", 1]]
SELECT "appointments".* FROM "appointments" WHERE "appointments"."patient_id" IN (?, ?, ?)  [["patient_id", 1], ["patient_id", 2], ["patient_id", 3]]

Caveats

There are a couple of caveats, though. Because we are now performing two queries instead of one, there may be performance implications. However, this would be the case regardless of whether you used the disable_joins feature or wrote the SQL manually since there are multiple queries across multiple databases happening. Additionally, since a join is longer being performed, a query with an order or limit will now be sorted in-memory. This is because the order from one table cannot be applied to another table. Finally, this setting must be added to all associations that you want joins to be disabled; obviously, Rails cannot simply guess when you do and don't want this feature enabled.

Conclusion

Being able to utilize the built-in association syntax that Rails provides without having to write custom SQL queries for objects that span multiple databases may not seem like a huge win, but it's not a trivial feature. For example, companies like GitHub have 30 databases configured in their Rails monolith, so being able to avoid maintaining all of that custom SQL likely saved them a considerable amount of engineering effort. You can read more about GitHub's use of cross-cluster associations here.

Further Reading

To learn more about cross-cluster associations, I recommend the following:

• PR that added disable_joins option to has_many relation
• PR that added disable_joins option to the has_one relation
• Official docs from Ruby on Rails about disable joins.