Honeybadger Developer Blog

Next.js vs React: What’s the difference and which should you use?

2026-05-28T07:00:00+00:00

The Next.js vs React question is not really a comparison between two competing tools — Next.js is built on top of React. React itself is a UI rendering JavaScript library used for building user interfaces across platforms, including web applications and mobile apps with React Native, while Next.js is a framework that wraps React and makes concrete decisions about routing, data fetching, and server-side concerns. Understanding this relationship is the starting point for every project decision you will make when building web applications.

React handles one job extremely well: taking a component tree and turning it into DOM output, then reconciling changes efficiently. Every other layer like how you fetch data, how you route between pages, what runs on the server versus the client is deliberately left to the developer or to third-party libraries. Next.js packages those decisions into a cohesive framework, adding server-side rendering, file-system-based routing, built-in image optimization, and an API routing layer that runs alongside your JavaScript code. This makes it particularly useful for complex projects that require coordination between the front-end and back-end.

This article covers what each tool does at a technical level, how they differ in behavior and file structure, how their rendering modes work, and answers the common question: what is Next.js vs React in practical terms?

What is React?

React's core contribution to web development is the component-based architecture combined with a virtual DOM diffing algorithm. Before React, updating the DOM in response to state changes meant either re-rendering entire page sections or writing granular imperative update logic that quickly became unmaintainable.

React introduced a declarative model: describe what the user interface should look like for a given state, and let the reconciler determine the minimum set of real DOM mutations required. This approach transformed web development workflows and made React a widely adopted JavaScript library for building complex user interfaces.

The component and hook model

A React component is a function that accepts props and returns JSX. The function re-runs whenever its state or props change, and React reconciles the output against the previous virtual DOM snapshot. Hooks like useState and useEffect let you attach stateful behavior and side effects to functional components without resorting to class syntax.

In practice, components are composed hierarchically, where child components receive data and callbacks from their parents. This composition model is what enables React to scale cleanly in complex projects, especially when managing deeply nested user interface trees.

// src/components/UserCard.tsx

import { useState, useEffect } from 'react';

interface User {
  id: number;
  name: string;
  email: string;
}
interface UserCardProps {
  userId: number;
}

export function UserCard({ userId }: UserCardProps) {
  const [user, setUser] = useState<User | null>(null);
  const [loading, setLoading] = useState(true);

  useEffect(() => {
    fetch(`/api/users/${userId}`)
      .then((res) => res.json())
      .then((data) => {
        setUser(data);
        setLoading(false);
      });
  }, [userId]);

  if (loading) return <div>Loading...</div>;
  if (!user) return <div>User not found</div>;

  return (
    <div className="card">
      <h2>{user.name}</h2>
      <p>{user.email}</p>
    </div>
  );
}

This component fetches a user when the userId prop changes, tracks the loading state, and renders conditionally based on that state. The useEffect dependency array [userId] ensures the fetch only re-runs when userId changes, not on every render. This pattern is idiomatic React, but notice that the data fetch happens entirely in the browser after the component mounts. There is no concept of running this on a server.

What React deliberately leaves out

React provides no routing system. Navigation between views requires installing a library like React Router or TanStack Router. React also has no built-in data fetching convention, no server-side rendering pipeline, no image optimization, and no API routes server. You can combine React with Express for server-side rendering and with third-party libraries like SWR or TanStack Query for caching, but you must assemble these pieces yourself across multiple JavaScript files.

This is not a weakness. For applications that run entirely in the browser, including dashboards and progressive web apps, the absence of framework opinions means less configuration overhead and more flexibility in your dependency choices. The cost is that you own the architecture decisions.

React also benefits from a large and active community, which means most problems you encounter already have established patterns or libraries.

What is Next.js and how does it extend React?

Next.js extends the React component model with conventions and runtime capabilities that React itself does not provide. The two most significant additions are the rendering pipeline (Server-Side Rendering (SSR), Static Site Generation (SSG), and Incremental Static Regeneration (ISR)) and the file-based routing system. Everything else—like API handling, image optimization, the Edge Runtime, font loading, and middleware—builds on top of those two foundations.

The App Router and Server Components

Next.js 13 introduced the App Router, which changed the default rendering model. In the App Router, every component is a React Server Component (RSC) unless you explicitly opt out with the 'use client' directive.

Server Components render on the server, can await async operations directly in the component body, and never ship their JavaScript code or the data-fetching JavaScript libraries they use to the client bundle. This results in automatic code splitting, where only the required interactive parts of the application are sent to the browser.

// app/users/[id]/page.tsx - runs on the server, zero client JS

interface PageProps {
  params: Promise<{ id: string }>;
}

async function getUser(id: string) {
  const res = await fetch(`https://api.example.com/users/${id}`, {
    next: { revalidate: 60 }, // ISR: revalidate this data every 60 seconds
  });
  if (!res.ok) throw new Error('Failed to fetch user');
  return res.json();
}

export default async function UserPage({ params }: PageProps) {
  const { id } = await params;
  const user = await getUser(id);
  return (
    <main>
      <h1>{user.name}</h1>
      <p>{user.email}</p>
    </main>
  );
}

This is fundamentally different from the React example above. The component is declared async and awaits the data directly, without useEffect, useState, or state management libraries. The fetch happens at request time on the server. The client receives pre-rendered HTML. The next: { revalidate: 60 } option activates Incremental Static Regeneration, so after the initial render, Next.js regenerates the page in the background when a request arrives after 60 seconds, serving the stale version until the fresh one is ready.

Client Components and the 'use client' boundary

When a component needs interactivity, you add 'use client' at the top of the file. This converts the component to a Client Component, which is hydrated in the browser. The boundary between Server and Client Components is explicit and composable: a Server Component can render interactive child components, but a Client Component cannot render a Server Component directly. This separation enables granular automatic code splitting and reduces bundle size.

// app/components/AddToCartButton.tsx
'use client';

import { useState } from 'react';

interface AddToCartButtonProps {
  productId: string;
  price: number;
}

export function AddToCartButton({ productId, price }: AddToCartButtonProps) {
  const [added, setAdded] = useState(false);

  async function handleClick() {
    await fetch('/api/cart', {
      method: 'POST',
      body: JSON.stringify({ productId, quantity: 1 }),
      headers: { 'Content-Type': 'application/json' },
    });
    setAdded(true);
  }

  return (
    <button onClick={handleClick} disabled={added}>
      {added ? 'Added to cart' : `Add to cart- $${price}`}
    </button>
  );
}

The product page itself (a Server Component) fetches product data on the server and renders HTML, while interactive child components handle user interactions like adding items to a cart. This pattern keeps user interfaces fast and lightweight. This granular control over the client bundle size is one of the most architecturally significant advantages Next.js provides over a pure React SPA (Single Page Application).

Key differences

The divergence between Next.js and a standalone React application becomes concrete when you compare how each handles routing, the rendering pipeline, project layout, search engine visibility, and runtime performance. These are not surface-level configuration differences; they reflect fundamentally different execution models.

Routing

React has no router. You install React Router and configure routes manually across multiple JavaScript files. Next.js uses file-based routing. Creating a file automatically registers a route. This removes boilerplate and makes it easier to scale routing in complex projects.

// src/main.tsx - React + React Router v6

import { createBrowserRouter, RouterProvider } from 'react-router-dom';
import { RootLayout } from './layouts/RootLayout';
import { HomePage } from './pages/HomePage';
import { ProductPage } from './pages/ProductPage';
import { NotFound } from './pages/NotFound';
import ReactDOM from 'react-dom/client';

const router = createBrowserRouter([
  {
    path: '/',
    element: <RootLayout />,
    errorElement: <NotFound />,
    children: [
      { index: true, element: <HomePage /> },
      { path: 'products/:id', element: <ProductPage /> },
    ],
  },
]);

ReactDOM.createRoot(document.getElementById('root')!).render(
  <RouterProvider router={router} />
);

Next.js uses a file-based routing system. Creating a file at app/products/[id]/page.tsx automatically registers the route /products/:id. The folder structure is the route definition. Layouts, loading states, error boundaries, and not-found pages are handled by special files (layout.tsx, loading.tsx, error.tsx, not-found.tsx) placed in the corresponding route segment directory.

app/
├── layout.tsx          → root layout, wraps all routes
├── page.tsx            → renders at /
├── loading.tsx         → Suspense fallback for /
├── error.tsx           → error boundary for /
├── products/
│   ├── page.tsx        → renders at /products
│   └── [id]/
│       ├── page.tsx    → renders at /products/:id
│       └── loading.tsx → Suspense fallback for /products/:id
└── api/
    └── cart/
        └── route.ts    → API endpoint at /api/cart

The file-system router removes an entire category of configuration work. You do not write route declarations, import pages manually, or manage a separate router configuration object. The trade-off is that your folder structure becomes load-bearing.

Rendering modes

A plain React application renders entirely in the browser. The server sends a mostly empty HTML document with a script tag, and React bootstraps the application in the client. This is client-side rendering (CSR). Next.js supports multiple rendering strategies and enables automatic code splitting at the route and component level. This significantly improves performance for large web applications. Search engines and users on slow connections both receive no meaningful content until the JavaScript parses, executes, and renders.

Next.js supports four rendering strategies, and you can mix them within a single application. Static site generation renders pages at build time, the HTML is computed once and served as a static file on every request.

Server-side rendering (SSR) renders on each request, allowing you to fetch fresh data per user or session. Incremental Static Regeneration (ISR) generates the page statically but revalidates it on a configurable interval. Client-side rendering is available for components marked with 'use client'.

// app/blog/[slug]/page.tsx

// Generate static paths at build time (static site generation)
export async function generateStaticParams() {
  const posts = await fetch('https://api.example.com/posts').then(r => r.json());
  return posts.map((post: { slug: string }) => ({ slug: post.slug }));
}

// Revalidate every 10 minutes (ISR)
export const revalidate = 600;

export default async function BlogPost({ params }: { params: Promise<{ slug: string }> }) {
  const { slug } = await params;
  const post = await fetch(`https://api.example.com/posts/${slug}`).then(r => r.json());
  return (
    <article>
      <h1>{post.title}</h1>
      <div dangerouslySetInnerHTML={{ __html: post.content }} />
    </article>
  );
}

The generateStaticParams function tells Next.js which slugs to pre-render at build time as part of static site generation. Setting revalidate = 600 on the module activates ISR. After ten minutes, the next request triggers a background regeneration. The page still serves instantly from cache while the fresh version is being built. This combination is the standard pattern for content-heavy sites: fast initial load time from static files, with eventual consistency as content changes.

SEO implications

Search engines index HTML content. A CSR React application sends an empty div; the content arrives only after JavaScript executes, which introduces indexing uncertainty and delays. Googlebot does execute JavaScript, but not instantly, and social media crawlers (Open Graph, Twitter Cards) typically do not execute JavaScript at all, meaning link previews for a CSR app will be blank.

Next.js pages rendered via server-side rendering or static site generation (SSG) deliver fully populated HTML on the initial response. You can set metadata like page titles, descriptions, Open Graph tags, and canonical URLs using the built-in Metadata API routes, which generate the correct <head> tags at render time on the server:

// app/products/[id]/page.tsx

import type { Metadata } from 'next';

interface PageProps {
  params: Promise<{ id: string }>;
}

export async function generateMetadata({ params }: PageProps): Promise<Metadata> {
  const { id } = await params;
  const product = await fetch(`/api/products/${id}`).then(r => r.json());
  return {
    title: `${product.name} - Acme Store`,
    description: product.description,
    openGraph: {
      title: product.name,
      images: [{ url: product.imageUrl }],
    },
  };
}

export default async function ProductPage({ params }: PageProps) {
  const { id } = await params;
  const product = await fetch(`/api/products/${id}`).then(r => r.json());
  return <ProductDetail product={product} />;
}

Note that the fetch in generateMetadata and the fetch in the page component both request the same URL. Next.js deduplicates these automatically using the built-in fetch cache. The network request is made once, even though you wrote it twice. This is a concrete example of the framework reducing accidental complexity.

Quick reference for React vs Next.js:

When to use React on its own

Standalone React (scaffolded with Vite or Create React App) is the appropriate choice when your application does not need server-side rendering, has no SEO requirements, and benefits from a thinner dependency footprint. The most common category is authenticated internal tooling: admin dashboards, CRM interfaces, data visualization tools, analytics platforms, and developer consoles. These applications sit behind a login screen, and the route structure is driven by application state (using state management libraries) rather than URL semantics.

React can also be extended beyond the web using React Native, allowing you to reuse concepts and patterns when building mobile apps.

A Vite-based React setup produces a minimal project with fast web development server startup and highly optimized production builds via Rollup. There is no server process to manage, no framework conventions to learn, and no build-time rendering pipeline to reason about.

If your team is familiar with React but not with Next.js-specific concepts like the App Router, Server Components, or the distinction between server-side and client-side data fetching, a plain React setup removes that cognitive surface area.

# Scaffold a React + TypeScript app with Vite
npm create vite@latest my-app -- --template react-ts
cd my-app
npm install

# Install Router for client-side navigation
npm install react-router-dom

# Install TanStack Query for data fetching and caching
npm install @tanstack/react-query

This combination of Vite, React Router, and TanStack Query covers the needs of most internal web applications. TanStack Query handles caching, background refetching, loading, and error states, among other things—with considerably less boilerplate than manually managing state with useEffect. For web applications where all data fetching is client-side anyway, this stack is competitive with Next.js in terms of developer experience.

Integrating React into an existing back-end

Another valid use of standalone React is when you have an existing server-side framework like Rails, Django, Laravel, or Spring, and you want to embed React components into specific pages rather than migrate to a JavaScript-first stack.

Third-party tools like Inertia.js let Rails or Laravel applications render React components server-side and manage navigation without a full SPA migration. In this architecture, Next.js would be redundant: the host framework already handles routing and server rendering.

When Next.js makes more sense

Next.js makes more sense when your application needs server-side rendering (SSR) for SEO or performance, when you want to colocate your API routes with your front-end code, or when you are building a content-heavy site where static site generation with revalidation provides both performance and freshness.

Public-facing applications with SEO requirements

Marketing sites, e-commerce websites, documentation, blogs, SaaS landing pages, and any application where pages need to rank in search results are natural fits for Next.js. Unlike React, the ability to combine static generation for high-traffic stable pages with ISR for frequently changing content, and server-side rendering (SSR) for personalized or session-dependent pages—all within a single codebase—is architecturally easy to implement.

Consider a product listing page on an e-commerce site. You want the page to load instantly (static or ISR for performance), include accurate metadata for search engines and social sharing (server-side rendering or static site generation (SSG) for SEO), and show personalized pricing or stock information (partial hydration with a Client Component making a client-side fetch after the static shell renders).

All of this is expressible in Next.js with standard patterns; in a plain React SPA, it requires either a separate server-side rendering (SSR) infrastructure or accepting the SEO limitations of client-side rendering.

Full-stack applications without a separate API service

Next.js Route Handlers let you define API endpoints that run alongside your front-end code, share type definitions, and deploy together as a single unit. For web development projects where the API and the UI are maintained by the same team and the back-end complexity does not justify a separate service, this is a significant simplification:

// app/api/orders/route.ts - runs on the server, not in the browser

import { NextRequest, NextResponse } from 'next/server';
import { db } from '@/lib/db'; // direct database access from the API route
import { auth } from '@/lib/auth';

export async function GET(request: NextRequest) {
  const session = await auth();
  if (!session) {
    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
  }

  const { searchParams } = new URL(request.url);
  const page = parseInt(searchParams.get('page') ?? '1', 10);
  const limit = 20;

  const orders = await db.order.findMany({
    where: { userId: session.user.id },
    orderBy: { createdAt: 'desc' },
    skip: (page - 1) * limit,
    take: limit,
  });

  return NextResponse.json({ orders, page });
}

This API route has direct database access, it imports a Prisma client, queries the database, and returns JSON. The front-end components in the same codebase can call this endpoint with a simple fetch('/api/orders'). TypeScript types for the response can be shared between the route handler and the calling component without publishing a separate package or running a code generation step. For small to medium teams building full-stack web applications, this tight coupling is a feature, not a liability.

Server Actions for form handling and mutations

Next.js Server Actions allow you to define server-side mutation functions that can be called directly from Client Components without going through an explicit REST endpoint. This is the pattern to reach for when you need form submissions, data mutations, or any user-triggered server-side operation:

// app/actions/createPost.ts
'use server';

import { revalidatePath } from 'next/cache';
import { db } from '@/lib/db';
import { auth } from '@/lib/auth';

export async function createPost(formData: FormData) {
  const session = await auth();
  if (!session) throw new Error('Unauthenticated');

  const title = formData.get('title') as string;
  const content = formData.get('content') as string;

  await db.post.create({
    data: {
      title,
      content,
      authorId: session.user.id,
    },
  });

  revalidatePath('/blog'); // purge the cached blog listing
}

// app/blog/new/page.tsx
'use client';

import { createPost } from '@/app/actions/createPost';

export default function NewPostForm() {
  return (
    <form action={createPost}>
      <input name="title" placeholder="Post title" />
      <textarea name="content" placeholder="Write your post..." />
      <button type="submit">Publish</button>
    </form>
  );
}

The form's action prop accepts the createPost server action directly. When submitted, Next.js serializes the form data and sends it to the server function, which executes in the Node.js runtime with full access to the database and environment variables. The revalidatePath call purges the Next.js cache for the /blog route, so the updated post list appears on the next request without a manual cache invalidation step. This entire pattern (form, mutation, cache invalidation) requires no custom API endpoint.

Next.js vs React: The architectural decision

The separation is cleaner than it might initially appear. React alone handles use cases where the application runs entirely in the browser and focuses on user interfaces through a component-based architecture. It's ideal for internal tools, authenticated dashboards, and single-page web applications without SEO requirements. This includes building interactive user interfaces for internal tooling, where client-side rendering is perfectly acceptable. Add Next.js when your web application needs to deliver server-rendered HTML for SEO or performance, when you want to run server-side logic alongside your front-end without maintaining a separate service, or when the file-system router and built-in optimizations justify the additional framework layer.

The practical signal to watch for is the nature of your first meaningful page render. If a blank loading screen is acceptable because the web application sits behind authentication and your users do not discover it via search, React's CSR model is okay. If search engine visibility is important, or the site needs to be fast for unauthenticated users on variable network connections, the server-side rendering capabilities of Next.js are important features.

One area worth monitoring is the Server Components model. The RSC architecture changes how you reason about data fetching and bundle size in ways that are not entirely settled. The mental model of interleaving server and client components, managing cache invalidation with revalidatePath and revalidateTag, and understanding which dependencies run only on the server has a meaningful learning curve. For developers new to both React and Next.js, starting with plain React as a JavaScript library and introducing Next.js only when you need server-side rendering or static site generation is a lower-risk path than beginning with the full App Router feature set.

If you liked this article and want to learn more about JavaScript, join the Honeybadger newsletter.

SIEM alerts: everything you need to know

2026-05-21T07:00:00+00:00

Let's walk through setting up SIEM (Security Information and Event Management) alerts to monitor security threats in applications. We will explain what SIEM alerts are, why they're relevant with regard to application security, and provide practical examples of common alerts a developer could implement. We will show how to configure simple alerts with Honeybadger Insights.

What is SIEM?

SIEM (Security Information and Event Management) refers to a class of security platforms that aggregate logs and security events from many systems and analyze them to detect threats. Just like in action movies where a thief exploits an unguarded angle, attackers in the cyber world look for weaknesses. A SIEM system works by pulling data from many different sources across the organization, including security system logs, usual and unusual network traffic, and threat intelligence. The SIEM analyzes them in one place to detect suspicious behavior instead of these signals living in silos.

The main aim of SIEM is to be a correlation engine. It doesn’t just collect raw data; it connects the dots using rule-based correlation, statistical analysis, and sometimes machine learning. The SIEM system continuously evaluates events in real time to identify patterns that indicate a real attack, enabling faster threat detection across your environment. Correlation helps prioritize alerts, not necessarily reduce them automatically. What this means is fewer false positives and clearer signals that something genuinely malicious is happening.

The importance of SIEM alerts

Organizations often run dozens or even hundreds of disconnected security tools where each generates alerts that require attention. Analysts are forced to jump between dashboards and manually investigate events. SIEM system addresses this problem by acting as the central nervous system of security operations. It prioritizes alerts by severity to help analysts immediately focus on incidents that pose the greatest risk.

The attackers don't even discriminate based on the size of the organisation. Some do it for the challenge or fun of it. They target organizations of all sizes and take advantage of openings across applications and cloud infrastructure. A SIEM system provides the visibility and context needed to detect these potential threats early, before they cause serious damage. Much like ignoring a staged diversion at the front of a museum and spotting the real break-in at the back, SIEM becomes one of the most powerful defensive tools security teams have.

How to respond to SIEM alerts effectively

Responding to SIEM alerts effectively determines whether a security incident is quickly contained or allowed to turn into a breach. Alert incident response procedures must be built on preparation, documentation, and practiced workflows. The goal is not just to react, but to respond effectively.

Establish runbooks for common alert types: Runbooks define exactly what should happen when a specific alert is triggered. For example, when a credential stuffing alert fires, the runbook should outline verification steps such as checking whether multiple accounts are affected, containment actions like blocking the attacking IP range, and notification requirements, including informing affected users. By standardizing incident responses, runbooks ensure consistency and help less-experienced team members handle incidents correctly without improvising under pressure.
Triage alerts immediately upon receipt: The first moments after receiving an alert are very important. Spend some time determining whether the alert represents a genuine threat or requires deeper investigation. Review recent similar alerts to see if the event is part of a broader attack pattern. You can also query the SIEM using BadgerQL (Honeybadger's Query Language) for additional context.
Document investigation steps and findings: Every alert investigation should leave a record. When alerts turn out to be false, document why they were triggered and whether tuning adjustments can prevent recurrence. When alerts uncover real attacks, record the attack vector, affected systems, and remediation steps taken.
Implement automated containment where safe: Certain scenarios, such as IP-based attacks, benefit from automated containment. When an IP triggers credential stuffing alerts, it can be temporarily blocked at the firewall or web application firewall. This allows containment to happen within seconds rather than waiting for manual action. However, automation must be used carefully. Over-automation can disrupt legitimate users.
Coordinate incident response across teams: Security incidents often span multiple layers, including applications, infrastructure, and databases. The security team may analyze data of the attack method, development teams patch vulnerable code, and security operations teams apply network-level blocks. Clear communication channels are essential. Many organizations rely on dedicated Slack channels or conference bridges to coordinate effectively during active incidents.

SIEM alert examples and types

SIEM alerts fall into several categories based on detection method and threat type. Each category serves a distinct purpose in the comprehensive security monitoring efforts to improve the security posture. Some identify known attack patterns, and some detect subtle behavioral deviations and enforce regulations. Below is a list of SIEM alerts covering the top SIEM alerts by threat detection method and also internal and external threat types.

Signature alerts: Signature-based alerts match specific patterns in log data, such as SQL injection attempts in HTTP requests or known malware file hashes. These alerts trigger when logs contain exact strings or regular expression matches associated with attacks. For example, detecting ' OR '1'='1' in query parameters signals a potential SQL injection probe.
Anomaly alerts: Anomaly-based alerts establish behavioral baselines and flag deviations. If a user account typically authenticates from New York during business hours, a login from Singapore at 3 AM exceeds normal behavior thresholds. These alerts require sufficient historical data to build accurate profiles.
Threshold alerts: Threshold-based alerts trigger when event counts exceed defined limits within time windows. Failed authentication attempts provide a clear example: five failed logins from a single IP address within ten minutes might indicate credential stuffing, while 100 failed logins across different accounts suggest a broader attack.
Compliance threat alerts: Compliance reporting enforces regulatory requirements and internal policies. PCI-DSS mandates alerts for unauthorized access attempts to cardholder data, while HIPAA requires notification of protected health information access outside normal workflows. Compliance frameworks require alerting and auditing, but tuning is still necessary to avoid alert fatigue.

Configuring alert triggers and correlation rules

Alert triggers define the conditions that generate notifications for common SIEM solution alerts. Simple triggers evaluate single log entries, while complex triggers correlate events across time windows and security data sources. Start with high-confidence signatures for known attacks before layering in anomaly detection and behavioral analysis.

Authentication alerts should trigger on multiple unsuccessful login attempts, successful logins following failed attempts (credential stuffing success), logins from blacklisted IP addresses, and authentications occurring simultaneously from geographically distant locations. Configure these with appropriate thresholds; three failed attempts might be a typo, but fifteen suggests an attack. Time windows matter too; five failures over 24 hours are less significant than five failures in sixty seconds.

Alerts on the application side monitor for injection attacks, path traversal attempts, and malicious file uploads. Web application firewalls generate logs that SIEM systems ingest and analyze. When requests attempt to access restricted file paths, or when uploaded files contain executable code. These alerts benefit from whitelisting safe patterns to reduce false positives from legitimate applications or user behavior.

Data access alerts flag unusual database queries, excessive record retrieval, and access to sensitive data outside normal application workflows. A user downloading sensitive data, such as customer records, at 2 AM warrants investigation, even if their credentials authenticate successfully. Configure these alerts to understand normal data access patterns.

Reducing false positives through SIEM alerts best practices

Reducing false positives is a core part of SIEM system alerts best practices, as excessive noise diminishes trust in alerting systems. Tuning requires iterative refinement based on investigation outcomes and environmental knowledge.

Whitelist known-safe activities that trigger alerts. Automated security scanners, monitoring systems, and internal tools often generate traffic patterns resembling attacks. Document these sources and exclude them from triggering alerts. For example, vulnerability scanners probe for SQL injection vulnerabilities as part of routine testing; their IP addresses should be whitelisted to prevent false alarms during scheduled scans.

Context development could add business intelligence to raw security events. An alert showing "100 failed login attempts from 203.0.113.45" provides limited context. Combining this with evolving threat intelligence reveals whether the IP belongs to a known botnet, including geolocation, which shows the attack origin, and correlating with past incidents indicates if this IP has targeted the organization previously.

Alert aggregation prevents duplicate notifications for the same incident. When an attacker probes multiple endpoints, each probe might trigger individual alerts. Aggregate these into a single incident showing the attack's scope rather than flooding the team with hundreds of similar notifications.

Managing alert fatigue and team burnout

Alert fatigue occurs when you receive so many notifications that they become desensitized, and you miss important security incidents. If your company receives hundreds of alerts a day, you're likely to get low investigation rates, with analysts ignoring the bulk of alerts.

Implement alert scoring that combines severity, context, and historical accuracy. Alerts that frequently lead to confirmed security incidents receive higher scores than those with poor signal-to-noise ratios. Machine learning models can predict which alerts warrant investigation based on key components like time of day, user reputation scores, and historical attack patterns. This scoring helps security analysts prioritize workloads when alert volumes exceed capacity.

Establish alert ownership and escalation paths. Each alert type needs a designated team responsible for investigation and remediation. Application security alerts route to security teams familiar with the codebase, infrastructure alerts go to operations, and access control alerts might escalate to the security team. Clear ownership prevents alerts from languishing in shared queues where everyone assumes someone else will investigate.

Setting up alerts with Honeybadger Insights

Applications generate a constant stream of events like failed logins, suspicious input, permission changes, and unexpected system or user behavior. Instead of sending raw log files into a traditional SIEM system pipeline and configuring complex agents, Honeybadger Insights is an observability tool that gives you structured security events directly from your application. These events become searchable, filterable, and alertable. This allows you to detect and respond to potential threats in real time without heavy infrastructure.

The idea is simple: treat security threats like application telemetry. Whenever something suspicious happens, you send a structured event to Honeybadger Insights. Then you create queries and alerts that behave like SIEM system rules.

For this guide, we will work with Nodejs. First, install the Honeybadger JavaScript package in your Node.js application. This example uses a basic Express server that reports failed login attempts.

npm init -y
npm install express @honeybadger-io/js

Create a file named server.js and configure Honeybadger.

const express = require("express");
const Honeybadger = require("@honeybadger-io/js");
const app = express();

app.use(express.json());

// Initialize Honeybadger
Honeybadger.configure({
  apiKey: process.env.HONEYBADGER_API_KEY,
  environment: "production",
});

// Simulated login endpoint
app.post("/login", (req, res) => {
  const { username, password } = req.body;

  // Fake authentication logic
  const isValid = username === "admin" && password === "secret";

  if (!isValid) {
    // Send a SIEM-style security event to Honeybadger Insights
    Honeybadger.event({
      type: "security.login.failed",
      user: username,
      ip: req.ip,
      timestamp: new Date().toISOString(),
      metadata: {
        reason: "Invalid credentials",
      },
    });

    return res.status(401).json({ error: "Invalid credentials" });
  }

  res.json({ message: "Login successful" });
});

// Example: suspicious input detection
app.post("/search", (req, res) => {
  const { query } = req.body;

  if (query && query.includes("' OR 1=1")) {
    Honeybadger.event({
      type: "security.sql_injection.detected",
      ip: req.ip,
      query,
      timestamp: new Date().toISOString(),
    });
  }

  res.json({ results: [] });
});

app.listen(3000, () => {
  console.log("Server running on port 3000");
});

In this setup, each security action is shown as a structured event. Instead of parsing logs later, Honeybadger stores these events in Insights, where they can be queried like a lightweight SIEM system dataset.

After events start flowing, you create SIEM-style alerts inside Honeybadger. For example, you can define a query such as:

type:"security.login.failed"

and configure an alert when the event count exceeds a threshold within a time window. This allows you to detect brute-force attacks, suspicious traffic spikes, or repeated injection attempts. You can also filter by IP address, user, environment, or any custom alert metadata you send.

To run the code locally, create a .env file or export your API key in the terminal:

export HONEYBADGER_API_KEY=your_api_key_here

Start the server:

node server.js

Then simulate events using curl or Postman:

curl -X POST http://localhost:3000/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"wrong"}'

Each failed request will appear in Honeybadger Insights within seconds. You can repeat the request multiple times to trigger your alert thresholds.

This approach turns your application into the primary source of security intelligence. Instead of forwarding system logs and building fragile parsing rules, you show events at the moment the risk occurs. The result is faster threat detection, cleaner security data, and SIEM-style security monitoring without the operational overhead of traditional security pipelines.

If you want to take this further, you can extend the pattern to permission changes, rate-limit violations, token misuse, or unusual API access patterns.

Benefits of Honeybadger for SIEM alerts

Traditional enterprise SIEM solutions require significant investment in licensing, infrastructure, and specialized personnel. Other security solutions like Splunk, QRadar, and ArcSight need you to have a dedicated security operations center and security teams trained in complex query languages. These platforms also require deep integration work to connect with your existing infrastructure and other security tools already in your stack.

That's not the case for Honeybadger compared to other tools used for security event management. It removes these barriers through developer-focused integration, so you focus more on improving security posture. Setup takes minutes, and installing the package requires just a few commands. The platform automatically captures events through existing error and performance monitoring instrumentation, which lets you enhance threat detection capabilities (improve security posture) without overhauling your entire toolchain.

Alert configuration happens through intuitive web interfaces. Creating a SIEM solution alert resembles configuring a performance threshold, which involves selecting the event pattern, defining the threshold, and specifying notification channels. Structured logging lets you query and analyze data immediately. Notifications integrate seamlessly through Slack, PagerDuty, and webhooks. The platform combines SIEM system capabilities with error tracking and performance monitoring in a single interface to provide a unified dashboard for security incident investigation.

You can sign up for a free trial of Honeybadger to see if this will fit your company’s needs.

Test-Commit-Revert: A useful workflow for testing legacy code in Ruby

2020-10-06T00:00:00+00:00

It happens to all of us. As software projects grow, parts of the production code we ship end up without a comprehensive test suite. When you take another look at the same area of code after a few months, it may be difficult to understand; even worse, there might be a bug, and we don't know where to begin fixing it.

Modifying production code without tests is a major challenge. We can't be sure if we'll break anything in the process, and checking everything manually is, at best, prone to mistakes; usually, it's impossible.

Dealing with this kind of code is one of the most common tasks we perform as developers, and many techniques have focused on this issue over the years, such as characterization tests, which we discussed in a previous article.

Today, we'll cover another technique based on characterization tests (test-commit-revert) and introduced by Kent Beck, who also introduced TDD to the modern programming world many years ago.

What's TCR?

TCR stands for "test, commit, revert", but it's more accurate to call it "test && commit || revert". Let's see why.

This technique describes a workflow to test legacy code. We'll use a script that will run the tests every time we save our project files. The process is as follows:

First, we create an empty unit test for the part of the legacy code we want to test.
We then add a single assertation and save the test.
Since we have our script set up, the test is automatically run. If it succeeds, the change is committed. If the test fails, the change is deleted (reverted), and we need to try again.

Once the test passes, we can then add a new test case.

Essentially, TCR (test, commit, revert) is about keeping your code in a "green" state instead of writing a failing test first (red) and then make it pass (green), as we do with test-driven development. If we write a failing test, it'll just vanish, and we'll be brought back to the "green" state again.

Purpose of test-commit-revert

The main goal of this technique is to understand the code a bit better each time you add a test case. This will naturally increase the test coverage and unblock many refactorings that, otherwise, wouldn't be possible.

One of the advantages of test, commit, revert is that it's useful in many scenarios. We can use it with production code that has no tests at all or with code that's partially tested. If the tests fail, we just revert the change and try again.

How can we use it?

Kent Beck shows, in different articles and videos (linked at the end), that a good approach is using a script that runs after certain files in the project are saved.

This will depend heavily on the project you're trying to test. Something like the following script, which is executed every time we save files with a plugin in the editor, is a good start:

(rspec && git commit -am "WIP") || git reset --hard

If you're using Visual Studio Code, a good plugin to execute on every save is "runonsave". You can include the above command or a similar one for your project. In this case, the whole config file would be

{
  "folders": [{ "path": "." }],
  "settings": {
    "emeraldwalk.runonsave": {
      "commands": [
        {
          "match": "*.rb",
          "cmd": "cd ${workspaceRoot} && rspec && git commit -am WIP || git reset --hard"
        }
      ]
    }
  }
}

Remember that later, you can squash the commit with Git directly in the command line or when merging the PR if you're using Github:

This means we'll only get one commit in the main branch for all the commits we did on the branch we're working on. This diagram from Github explains it well:

Writing our first test with TCR

We'll use a simple example to illustrate the technique. We have a class that we know is working, but we need to modify it.

We could just make a change and deploy the changes to production. However, we want to be sure that we don't break anything in the process, which is always a good idea.

# worker.rb
class Worker
  def initialize(age, active_years, veteran)
    @age = age
    @active_years = active_years
    @veteran = veteran
  end

  def can_retire?
    return true if @age >= 67
    return true if @active_years >= 30
    return true if @age >= 60 && @active_years >= 25
    return true if @veteran && @active_years > 25

    false
  end
end

The first step would be to create a new file for the tests, so we can start adding them there. We've seen the first line in the can_retire? method with

  def can_retire?
    return true if @age >= 67
    ...
    ...
  end

Thus, we can test this case first:

# specs/worker_spec.rb
require_relative './../worker'

describe Worker do
  describe 'can_retire?' do
    it "should return true if age is higher than 67" do

    end
  end
end

Here's a quick tip: when you're working with test, commit, revert, every time you save, the latest changes will disappear if the tests fail. Therefore, we want to have as much code as possible to "set up" the test before actually writing and saving the line or lines with the assertion.

If we save the above file like that, we can then add a line for the test.

require_relative './../worker'

describe Worker do
  describe 'can_retire?' do
    it "should return true if age is higher than 67" do
      expect(Worker.new(70, 10, false).can_retire?).to be_true ## This line can disappear when we save now
    end
  end
end

When we save, if the new line doesn't vanish, we've done a good job; the test passes!

Adding more tests

Once we have our first test, we can keep adding more cases while taking into account false cases. After some work, we have something like this:

# frozen_string_literal: true

require_relative './../worker'

describe Worker do
  describe 'can_retire?' do
    it 'should return true if age is higher than 67' do
      expect(Worker.new(70, 10, false).can_retire?).to be true
    end

    it 'should return true if age is 67' do
      expect(Worker.new(67, 10, false).can_retire?).to be true
    end

    it 'should return true if age is less than 67' do
      expect(Worker.new(50, 10, false).can_retire?).to be false
    end

    it 'should return true if active years is higher than 30' do
      expect(Worker.new(60, 31, false).can_retire?).to be true
    end

    it 'should return true if active years is 30' do
      expect(Worker.new(60, 30, false).can_retire?).to be true
    end
  end
end

In every case, we write the "it" block first, save, and then add the assertion with expect(...).

As usual, we can add as many tests as possible, but it makes sense to avoid adding too many once we're relatively sure that everything is covered.

There are still a few cases to cover, so we should add them just for completeness.

Final tests

Here's the spec file in its final form. As you can see, we could still add more cases, but I think this is enough to illustrate the process of TCR.

# frozen_string_literal: true

require_relative './../worker'

describe Worker do
  describe 'can_retire?' do
    it 'should return true if age is higher than 67' do
      expect(Worker.new(70, 10, false).can_retire?).to be true
    end

    it 'should return true if age is 67' do
      expect(Worker.new(67, 10, false).can_retire?).to be true
    end

    it 'should return true if age is less than 67' do
      expect(Worker.new(50, 10, false).can_retire?).to be false
    end

    it 'should return true if active years is higher than 30' do
      expect(Worker.new(60, 31, false).can_retire?).to be true
    end

    it 'should return true if active years is 30' do
      expect(Worker.new(20, 30, false).can_retire?).to be true
    end

    it 'should return true if age is higher than 60 and active years is higher than 25' do
      expect(Worker.new(60, 30, false).can_retire?).to be true
    end

    it 'should return true if age is higher than 60 and active years is higher than 25' do
      expect(Worker.new(61, 30, false).can_retire?).to be true
    end

    it 'should return true if age is 60 and active years is higher than 25' do
      expect(Worker.new(60, 30, false).can_retire?).to be true
    end

    it 'should return true if age is higher than 60 and active years is 25' do
      expect(Worker.new(61, 25, false).can_retire?).to be true
    end

    it 'should return true if age is 60 and active years is 25' do
      expect(Worker.new(60, 25, false).can_retire?).to be true
    end

    it 'should return true if is veteran and active years is higher than 25' do
      expect(Worker.new(60, 25, false).can_retire?).to be true
    end
  end
end

Ways to refactor

If you've read this far, there's probably something that feels a bit off with the code. We have many "magical numbers" that should be extracted into constants, both in the test and in the Worker class.

We could also create private methods for each case in the main can_retire? public method.

I'll leave both potential refactorings as exercises for you. However, we have tests now, so if we make a mistake in any step, they will tell us.

Where do you go from here?

I encourage you to try test-commit-revert with your projects and production code. It's a very cheap experiment because you don't need any fancy continuous integration in an external server or a dependency with a new library. All you need is a way to execute a command every time you save certain files on your computer.

It'll also give you a "gaming" experience when adding tests, which is always fun and interesting. Additionally, the discipline of having failing tests removed from your editor (a safety measure that kicks in whenever tests fail) will give you an extra safety net by confirming that the tests you're pushing to the repository are actually passing.

I hope you find this new technique useful when dealing with legacy code. I've used multiple times in the last few months, and it's always been a pleasure.

Tips for upgrading Python/Django versions in existing apps

2023-07-20T00:00:00+00:00

Python is a robust and powerful programming language. In addition to machine learning, Python can be used for tasks such as web scraping, image processing, scientific computing, and much more. A framework such as Django, which is built on top of Python, enables you to build beautiful web applications—top websites such as Dropbox, Instagram, and YouTube use Django.

However, as you create applications and release them to consumers, upgrading to the latest Python version becomes essential to keep pace with updates and new features. For instance, Python 3.11 was released in October 2022, sporting various bug fixes. Before that, there was Python 3.10, 3.9, 3.8, and others. There are also numerous third-party libraries that you should regularly watch out for.

Incorporating these updates into our applications can be beneficial, as doing so can enhance the security and reliability. However, doing this incorrectly could also break your application.

Why you should be upgrading your Python version

Upgrading your Python and Django applications to the latest Python version is important for several reasons.

New Python versions come with additional features and improvements. Newer Django versions can help reduce boilerplate code, enabling you to develop and push products to the market much faster. They can also help you learn new ways of adding functionalities to your application that can be more fun and easier to implement.
New Python versions can help with bug fixes in your code. Apart from enabling your application to behave as expected, eliminating bugs also streamlines the software development process, meaning that you'll be less frustrated. A Python upgrade can improve your app's stability during production.
Upgrading your application also makes it easier to maintain your codebase. You can quickly incorporate changes in your project without major downtimes.
Ensuring you're using the latest Python version and Django version also improves security. Hackers are always looking for new ways to infiltrate systems, steal data, install malicious files and viruses, and just wreak havoc. Upgrading your application to use the latest Python version or Django ensures that you have the latest security updates.

Now that we understand why upgrading applications is necessary, let's jump into the action itself.

Things to know before updating

Before updating the Python version, you should read the changes in the newest Python release. Go through the official documentation, particularly the release notes. Note that with each new version, Django releases accompanying documentation, which is quite helpful when upgrading applications.

Next, look at the functions, objects, and other items deprecated in the latest Python version. Consider the deadline or timeline in which these deprecation changes are set to come into effect. Understanding the deprecation timeline allows you to better plan your upgrade process, including the prioritization of certain tasks over others.

You should also keep an eye on backwards compatibility. Remember that some things that work in your existing application may fail when you upgrade to a new Python version. This is why it's recommended to test things first before pushing to production. Whenever an upgrade causes your app to break, you can roll back to the previous Python version as you figure out the next steps.

How to update Python/Django apps

In this section, we will learn how to upgrade Python and Django in a simple Django Movie API to use a newer Python version. You can download the project's code from this GitHub repository.

Check Django and other library versions

Once you've downloaded the Movie API project and finished setting it up on your computer, the first step is to check the versions of the dependencies that the project uses.

To do this, launch the project in your preferred code editor and open the requirements.txt file. All dependencies used by the project are listed in this file, as follows.

asgiref==3.3.1
Django==3.1.8
django-filter==2.4.0
djangorestframework==3.12.2
djangorestframework-simplejwt==4.6.0
pytz==2021.1
sqlparse==0.4.1

Since we now know the versions in use (shown above), the next step is to determine the current official version from the documentation. For instance, the latest official Django version is 4.2.1; our application uses an older version (3.1.8), so there's a need to upgrade.

Activate local debugging

Trying to update things in production could cause them to break and lead to poor user experience and other negative consequences. You should test your applications locally and ensure that everything is working before pushing them to production.

You can activate local debugging by setting the debug option in the settings.py file to True.

# settings.py
import os

# Build paths inside the project like this: os.path.join(BASE_DIR, …)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/wa

# SECURITY WARNING: Don’t run with debug turned on in production!
DEBUG = True  # Set debug to true

ALLOWED_HOSTS = []

Run Django checks with python -Wall manage.py check

Before starting the upgrade, we'll need to run several checks to determine if there are any deprecation warnings in our project.

Deprecation warnings show that certain features will stop working at some point simply because they have been replaced by better alternatives.

You can check for any warnings using the following command:

python -Wa manage.py test

Install new dependencies

Now that we have enabled local debugging and identified any deprecation warnings in our project, it's time to install new dependencies in our project.

To get the latest Django version, we use the following command.

python -m pip install -U Django

When you run the above command, the following output should appear in your terminal. We have now successfully updated Django to ver 4.2.1, asgiref to ver 3.6.0, and tzdata to ver 2023.3.

Requirement already satisfied: Django in c:\users\wanjamike\appdata\local\programs\python\python310\lib\site-packages (3.1.8)
Collecting Django
  Downloading Django-4.2.1-py3-none-any.whl (8.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 43.5 kB/s eta 0:00:00
Requirement already satisfied: sqlparse>=0.3.1 in c:\users\wanjamike\appdata\local\programs\python\python310\lib\site-packages (from Django) (0.4.1)
Collecting tzdata
  Using cached tzdata-2023.3-py2.py3-none-any.whl (341 kB)
Collecting asgiref<4,>=3.6.0
  Using cached asgiref-3.6.0-py3-none-any.whl (23 kB)
Installing collected packages: tzdata, asgiref, Django
  Attempting uninstall: asgiref
    Found existing installation: asgiref 3.3.1
    Uninstalling asgiref-3.3.1:
      Successfully uninstalled asgiref-3.3.1
  Attempting to uninstall: Django
    Found existing installation: Django 3.1.8
    Uninstalling Django-3.1.8:
      Successfully uninstalled Django-3.1.8
Successfully installed Django-4.2.1 asgiref-3.6.0 tzdata-2023.3

We can update the remaining dependencies in our project using the following command:

python -m pip install -U django-filter djangorestframework djangorestframework-simplejwt pytz sqlparse

Expected output:

Successfully installed django-filter-23.2 djangorestframework-3.14.0 djangorestframework-simplejwt-5.2.2 pytz-2023.3 sqlparse-0.4.4

Let's generate a new requirements.txt file and double-check that we have the latest libraries in our project.

pip freeze > requirements.txt

When you open the requirements.txt, you should find all the latest dependencies, as shown below:

asgiref==3.6.0
Django==4.2.1
django-filter==23.2
djangorestframework==3.14.0
djangorestframework-simplejwt==5.2.2
pytz==2023.3
sqlparse==0.4.4
tzdata==2023.3

Check methods and functions

In most software updates, developers will try to change how certain functions or components work to improve performance, efficiency, memory usage, or convenience. Rather than fighting such changes, consider embracing and integrating them into your application to realize their benefits.

Note that while this may be a time-consuming step, the benefits are worth it. You should go through the official documentation to understand new changes and learn how you can integrate them into your application.

Let's review some of the changes in Django 4.2 that can help you in future upgrades.

First, support for MariaDB 10.3, MySQL 5.7, and PostgreSQL 11 databases was dropped. If your application was using any of these database versions, consider upgrading. Make sure to back up your data before upgrading to avoid service disruptions.

Second, Django 4.2 changed how we do data indexing. In the past, we indexed data using the statement below:

index_together = [["rank", "name"]]

But now we must call the Index method from the models class, as demonstrated below.

indexes = [models.Index(fields=["rank", "name"])]

Third, the length_is template filter is now deprecated and has instead been replaced by the traditional == operator.

Don't do this:

{% if value|length_is:4 %}…{% endif %}
{{ value|length_is:4 }}

Do this:

{% if value|length == 4 %}…{% endif %}
{% if value|length == 4 %}True{% else %}False{% endif %}

Updating Python is important

Updates are a normal thing during software development. Therefore, it's best to learn how to update your app's Python version and integrate changes in our applications. Apart from improved security, software updates allow us to deal with bugs and ensure that our code is maintainable.

When upgrading to a new version of Python, make sure to test things out in your local environment before pushing them to production. We don't want things to break and cause a poor user experience.

Errors in Python: types, causes, and examples

2026-04-27T00:00:00+00:00

Errors in Python are issues in a program that cause incorrect results or prevent proper execution. Some Python errors are loud and obvious, and your code barely gets started before it throws an error that tells you exactly what went wrong. Other errors are more subtle, allowing your Python program to run without complaints while silently producing incorrect results that only become apparent later. These differences become clearer when you group errors in Python based on how they occur and how they impact execution.

For example, syntax errors get caught before the code even runs, and runtime errors blow up mid-execution. System-level errors have nothing to do with your code—and they still stop execution—while logical errors don’t stop execution but produce incorrect results. Understanding the different types of errors and learning how to avoid them is essential for writing reliable and robust code. Let's look at the different types of errors in Python, their causes, and how to avoid them.

Different types of errors in Python

We can broadly categorize Python errors into four types, as shown in the following image:

Syntax errors: Syntax errors in Python occur due to invalid syntax, incorrect indentation, or typos. These errors are detected before the program is executed, while the Python interpreter parses the code.
Runtime errors: In Python, runtime errors occur during execution when the program encounters an invalid operation.
System-level errors: System-level errors in Python are raised by the runtime environment or the operating system due to issues such as memory overflow or interruptions.
Logical errors: Logical errors occur when the program's logic is incorrect, even though the code runs without errors, producing incorrect results.

Let's discuss all these Python error types one by one in detail with examples, starting with syntax errors.

Syntax errors in Python

Syntax errors are the errors caused by invalid code structure, typos, incorrect indentation, etc. For example, an if block is defined in Python using the if keyword, a boolean condition, and the : character. If we skip : while writing an if block in Python, the code runs into SyntaxError with the error message SyntaxError: expected ':', as shown in the following example:

if x > 10
    print("Honeybadger")

This code gives the following error message:

  File "/path/to/code.py", line 1
    if x > 10
             ^
SyntaxError: expected ':

Syntax errors occur when the interpreter cannot parse the code because it violates Python’s grammar rules. Let's discuss some of the common syntax errors in Python.

Indentation errors in Python

Python uses spaces and tabs for code indentation. The code will run into IndentationError if a code block doesn't have correct indentation. For example, defining an if block requires us to indent the lines in the if block to the right by two/four spaces. If we don't indent the lines in the if block, the code runs into IndentationError with the error message IndentationError: expected an indented block after 'if' statement on line x.

if x > 10:
print("Honeybadger")

The unindented if block gives the following error:

  File "/path/to/code.py", line 2
    print("Honeybadger")
    ^
IndentationError: expected an indented block after 'if' statement on line 1

Similarly, we need to indent the code block inside a function definition. Not doing so gives us an IndentationError with the message IndentationError: expected an indented block after function definition on line x, as shown below:

def say_hello(name):
print(f"Hi {name}, you are at Honeybadger")

In this code, the print statement inside the say_hello() function isn't indented to the right. Hence, the code gives the following error:

  File "/path/to/code.py", line 2
    print(f"Hi {name}, you are at Honeybadger")
    ^
IndentationError: expected an indented block after function definition on line 1

When you indent a code block, it is important to keep the indentation the same for each statement in the code block. Otherwise, the program runs into IndentationError. For example, if you indent the first statement of a code block by four spaces and the second statement by two spaces, the program will run into IndentationError with the error message IndentationError: unindent does not match any outer indentation level, as shown in the following example:

def say_hello(name):
    print(f"Hi {name}, you are at Honeybadger")
  print("Great seeing you here.")

Output:

  File "/path/to/code.py", line 3
    print("Great seeing you here.")
                                   ^
IndentationError: unindent does not match any outer indentation level

Similarly, if you indent the first statement in a code block by two spaces and the second statement by four spaces, the program will run into IndentationError with the message IndentationError: unexpected indent, as shown below:

def say_hello(name):
  print(f"Hi {name}, you are at Honeybadger")
    print("Great seeing you here.")

Output:

  File "/path/to/code.py", line 3
    print("Great seeing you here.")
IndentationError: unexpected indent

Tab error

TabError is a specific indentation error caused by mixing tabs and spaces to indent code blocks. For instance, you can indent a statement by the same distance using four spaces or one tab. Visually, it looks the same. However, if you indent a statement in a code block using a tab and another using four spaces, the program will run into TabError with the error message TabError: inconsistent use of tabs and spaces in indentation, as shown in the following example:

def say_hello(name):
    print(f"Hi {name}, you are at Honeybadger") # Indentation using Tab
    print("Great seeing you here.")  # Indentation using four spaces

Output:

  File "/path/to/code.py", line 3
    print("Great seeing you here.")  # Indentation using four spaces
TabError: inconsistent use of tabs and spaces in indentation

Python 3 explicitly disallows mixing tabs and spaces for indentation in a way that makes the meaning ambiguous, and you should always avoid it.

Unclosed strings/ brackets/ parentheses

A Python program runs into a SyntaxError if you don't close a string, parentheses, or a bracket. For example, if you don't close a string, the program runs into SyntaxError with the message SyntaxError: unterminated string literal.

if x > 10:
    print("Honeybadger)

In this code, the closing " is missing in the "Honeybadger string. Due to this, the program runs into SyntaxError, as shown below:

  File "/path/to/code.py", line 2
    print("Honeybadger)
          ^
SyntaxError: unterminated string literal (detected at line 2)

Similarly, if we forget the closing parenthesis while calling a function or defining a tuple, the program runs into SyntaxError with the message SyntaxError: '(' was never closed, as shown below:

print("Honeybadger"

Output:

  File "/path/to/code.py", line 1
    print("Honeybadger"
         ^
SyntaxError: '(' was never closed

Just like parentheses, if we miss the closing bracket while defining a list, the program runs into SyntaxError with the message SyntaxError: '[' was never closed, as shown below:

my_list = [1, 2, 3

Output:

  File "/path/to/code.py", line 1
    my_list = [1, 2, 3
              ^
SyntaxError: '[' was never closed

Invalid assignment errors

Invalid assignment errors are mostly caused by assigning values to literals or function calls. For example, if we assign the value "Honeybadger" to a string literal "name", the program throws a SyntaxError with the message SyntaxError: cannot assign to literal here. Maybe you meant '==' instead of '='?, as shown below:

"name" = "Honeybadger"

Output:

  File "/path/to/code.py", line 1
    "name" = "Honeybadger"
    ^^^^^^
SyntaxError: cannot assign to literal here. Maybe you meant '==' instead of '='?

Similarly, if we assign a value to a Python keyword, the program runs into a SyntaxError. For example, assigning the value Honeybadger to a variable class results in a SyntaxError with the message SyntaxError: invalid syntax, as class is a Python keyword.

class = "Honeybadger"

Output:

  File "/path/to/code.py", line 1
    class = "Honeybadger"
          ^
SyntaxError: invalid syntax

An assignment error also occurs when we miss a = character while comparing values using the equality operator. For example, if we use = instead of == to compare two values, the program runs into SyntaxError with the error message SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?, as shown below:

name = "Honeybadger"
input_string = "Honeybadger"
if name = input_string:
  print(name)

Output:

  File "/path/to/code.py", line 3
    if name = input_string:
       ^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?

How to avoid syntax errors in Python?

Syntax errors occur due to incorrect indentation, mismatched delimiters, missing punctuation, invalid variable names, or incorrect operators. You can avoid syntax errors using the following best practices:

Always add the required colon : after block statements like if, for, while, def, and class.
Avoid using reserved keywords such as class, for, if, or return as variable names.
Always close all the parentheses (), brackets [], and braces {}.
Close all string literals with matching quotation marks ' or ".
Follow Python syntax rules and ensure statements are written in the correct format.
Maintain consistent indentation, preferably using 4 spaces per indentation level, and do not mix tabs and spaces for indentation.

In addition to the above practices, you can use IDEs or code editors such as PyCharm, Spyder, or VS Code that provide syntax highlighting and linting to detect syntax errors early.

Runtime errors in Python

Runtime errors occur in a Python program after it passes the syntax check and starts executing, when something goes wrong. Examples of runtime errors in Python include ZeroDivisionError, NameError, TypeError, and ValueError. Let’s discuss the different runtime errors, their causes, and ways to avoid them.

Zero division error

The ZeroDivisionError exception is one of the most common arithmetic errors that occurs if the denominator of a division operation is zero.

x = 10 / 0

Here, we are dividing 10 by 0. Hence, the program runs into ZeroDivisionError with the error message ZeroDivisionError: division by zero:

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    x = 10 / 0
ZeroDivisionError: division by zero

Name error

The NameError exception is a runtime error that occurs when a variable is referenced before assignment.

y = x / 10

In this code, we tried to divide x by 10 without defining x or assigning it a value. Hence, the variable name x isn't present in the scope of the program, and the program runs into a NameError exception with the error message NameError: name 'x' is not defined.

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    y = x / 10
NameError: name 'x' is not defined

The NameError exception also occurs if you use a variable first and define it later in the program. For example, consider the following code:

print(greeting)
greeting = "Hi, you are at Honeybadger"

Here, we have referenced the variable greeting and later assigned it a value. Hence, the program throws a NameError exception.

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    print(greeting)
NameError: name 'greeting' is not defined

Unbound local error

The UnboundLocalError exception occurs when a local variable is referenced before assignment, in a function or method. For instance, consider the following code:

def say_hello():
    print(greeting)
    greeting = "Hi, you are at Honeybadger"
    
say_hello()

In this code, we have referenced the variable greeting before assigning it a value in the say_hello() function. When we call the say_hello() function, the program raises the UnboundLocalError exception with the message UnboundLocalError: local variable 'greeting' referenced before assignment.

Traceback (most recent call last):
  File "/path/to/code.py", line 5, in <module>
    say_hello()
  File "/path/to/code.py", line 2, in say_hello
    print(greeting)
UnboundLocalError: local variable 'greeting' referenced before assignment

Here, if we hadn't assigned any value to the variable after the print statement, the program would have run into a NameError exception, as shown below:

def say_hello():
    print(greeting)
    print("Hi, you are at Honeybadger")

say_hello()

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 5, in <module>
    say_hello()
  File "/path/to/code.py", line 2, in say_hello
    print(greeting)
NameError: name 'greeting' is not defined

Type error

The TypeError exceptions in Python occur when an operation is applied to a value or variable of an incompatible data type. For example, adding an integer and a string results in a TypeError exception with the error message TypeError: unsupported operand type(s) for +: 'int' and 'str', as shown below:

x = 10 + "Honeybadger"

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    x = 10 + "Honeybadger"
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Similarly, calling a non-callable object or iterating on a non-iterable object also results in a TypeError exception, as shown below:

name = "Honeybadger"
name()

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    name()
TypeError: 'str' object is not callable

In this code, we defined a string variable name and tried to use it as a function in the second line. Hence, the program throws a TypeError exception with the message TypeError: 'str' object is not callable.

Value error

The ValueError exception occurs in a Python program when we use an input or a variable with the correct data type but an inappropriate value. For instance, the int() function converts a string to an integer. If the string passed to the int() function cannot be converted into an integer, the program runs into a ValueError exception, as shown in the following example:

x = int("10")
y = x + int("Honeybadger")
print(y)

In this code, the statement x=int("10") executes successfully because "10" is successfully converted into an integer. However, the string "Honeybadger" cannot be converted to an integer. Hence, the second line of the code raises a ValueError exception with the error message ValueError: invalid literal for int() with base 10: 'Honeybadger', as follows:

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    y = x + int("Honeybadger")
ValueError: invalid literal for int() with base 10: 'Honeybadger'

Similarly, square roots aren't defined for negative numbers. Hence, passing a negative number to the math.sqrt() function results in a ValueError exception due to an inappropriate value.

import math
x = math.sqrt(-10)

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    x = math.sqrt(-10)
ValueError: math domain error

In this code, -10 has the correct integer data type that the sqrt() function requires. However, it is an invalid value, and we get a ValueError exception with the message ValueError: math domain error.

Index error

The IndexError exception occurs in a Python program when we try to access an element at an index that doesn't exist in an iterable object like a string, list, or tuple. For example, if a list has six elements and we try to access the element at index 6 (the seventh element), the program runs into an IndexError exception with the message IndexError: list index out of range, as shown below:

my_list = [1, 2, 3, 4, 5, 6]
print(my_list[6])

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    print(my_list[6])
IndexError: list index out of range

Similarly, if we try to access an element at a non-existent index in a string, the program runs into an IndexError exception with the message IndexError: string index out of range, as shown in the following code:

name = "Honeybadger"
print(name[20])

In this code, we tried to access the character at index 20 in the string. However, the string is eleven characters long. Hence, the program raises an IndexError exception.

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    print(name[20])
IndexError: string index out of range

Key error

KeyError exceptions occur when we try to access a non-existent key in a Python dictionary. For instance, the dictionary in the following code has keys "a", "b", "c", and "d". When we try to fetch a value with the key "e", the program runs into a KeyError exception, as shown below:

my_dict = {"a": 1, "b": 2, "c": 3, "d": 4}
print(my_dict["e"])

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    print(my_dict["e"])
KeyError: 'e'

Module not found error

The ModuleNotFoundError occurs when we try to import a module that hasn't already been installed or downloaded to the Python module search path.

For example, suppose you want to use Honeybadger for error monitoring in a Python application. However, if you don't install honeybadger using pip and start directly by importing the honeybadger module into your code, the program will run into ModuleNotFoundError with the message ModuleNotFoundError: No module named 'honeybadger'.

import honeybadger
print("You are at Honeybadger")

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    import honeybadger
ModuleNotFoundError: No module named 'honeybadger'

ModuleNotFoundError is a specific type of ImportError that occurs when Python cannot find the module file being imported.

Import errors in Python

ImportError is a more general exception that can occur for various reasons during the import process, even when the module file exists but cannot be imported successfully due to dependency requirements or other issues. If a module exists but raises an exception while being imported, Python raises ImportError.

For instance, if we try to import a non-existent function from the honeybadger module after installing it, the program runs into an ImportError exception.

from honeybadger import nonexistingfunction

Here, we tried to import nonexistingfunction from the honeybadger module. Hence, the program raises an ImportError with the message ImportError: cannot import name 'nonexistingfunction' from 'honeybadger':

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    from honeybadger import nonexistingfunction
ImportError: cannot import name 'nonexistingfunction' from 'honeybadger'

Attribute error

In Python, every object has a set of associated attributes, i.e., field names and methods. For example, a Python list has the append() method that we use to add new values to a list. However, a tuple, an integer, a string, or a floating-point value doesn't have the append() method. Hence, if we invoke the append() method on a tuple, the program raises an AttributeError exception.

my_tuple = (1, 2, 3, 4, 5)
my_tuple.append(6)

In this code, we have used the append() method on a tuple. Hence, the program throws an AttributeError exception with the message AttributeError: 'tuple' object has no attribute 'append'.

Traceback (most recent call last):
  File "/path/to/code.py", line 2, in <module>
    my_tuple.append(6)
AttributeError: 'tuple' object has no attribute 'append'

Memory error

MemoryError in Python is a built-in exception that occurs when a program fails to allocate memory for a Python object. It usually occurs when handling extremely large datasets, constructing oversized data structures, or running inefficient code that leads to excessive memory consumption or memory leaks.

For example, creating a huge list having ten billion elements can lead to a MemoryError exception, as shown below:

my_list = [10] * (10**10)

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    my_list = [10] * (10**10)
MemoryError

While MemoryError is a runtime error, it often indicates that the program has exceeded a system limit, such as running out of RAM or exceeding the operating system's address space limit.

Recursion error

A recursion error is a runtime error that occurs when the recursion depth exceeds the limit of 1000 recursive calls. The RecursionError exception occurs if we forget to add a base case or a terminating condition while defining a recursive function. For instance, consider the following increment_till_hundred() function:

def increment_till_hundred(x):
    x += 1
    print(x)
    increment_till_hundred(x)
    
increment_till_hundred(80)

In the increment_till_hundred() function, we haven't defined any termination condition if the value of x reaches 100. Hence, the function keeps making the recursive call, exceeding the limit of 1000 recursive calls, and the program runs into RecursionError with the message RecursionError: maximum recursion depth exceeded while calling a Python object, as shown below:

Traceback (most recent call last):
  File "/path/to/code.py", line 6, in <module>
    increment_till_hundred(80)
  File "/path/to/code.py", line 4, in increment_till_hundred
    increment_till_hundred(x)
  [Previous line repeated 994 more times]
  File "/path/to/code.py", line 3, in increment_till_hundred
    print(x)
RecursionError: maximum recursion depth exceeded while calling a Python object

How to avoid runtime errors in Python?

Runtime errors are difficult to detect because they do not prevent the program from starting execution, unlike syntax errors. Hence, the program runs normally at first, and the error may only appear later when the line containing the problematic code is executed. To avoid runtime errors, you can use the following best practices:

Always validate the input data before processing to ensure it has the correct data type, format, and range.
Always implement exception handling in your Python code to catch and handle runtime errors gracefully, rather than crashing the program.
Perform type checking using the isinstance() function or convert the data type of values using int(), float(), and str() functions before applying operations on values.
Maintain a properly configured runtime environment, ensuring all required modules, dependencies, and system libraries are correctly installed.
Test the code with different edge cases to identify potential runtime failures early.

Apart from the above practices, always write modular, well-structured code so you can easily isolate and debug errors if they occur.

System-level errors in Python

System-level errors occur in a Python program when it encounters issues such as I/O failures, memory overflows, or connection errors. All the system-level errors in Python are raised using the OSError exception or its subclasses. Let's discuss the different system-level errors in Python and how to avoid them.

File not found errors in Python

The FileNotFoundError exception occurs when we try to read a non-existent file. For example, suppose that we want to read a text file named sampletextfile.txt using the open() function. If the file doesn't exist, the program runs into FileNotFoundError with the message FileNotFoundError: [Errno 2] No such file or directory: 'sampletextfile.txt', as shown below:

file = open("sampletextfile.txt", "r")

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    file = open("sampletextfile.txt", "r")
FileNotFoundError: [Errno 2] No such file or directory: 'sampletextfile.txt'

Permission error

If a file exists and we don't have permission to read or modify it, the program raises a PermissionError exception. For example, suppose that we have a file samplefile.txt with only read access, as shown in the image:

Now, if we try to open the file in append mode and modify it, the program raises a PermissionError exception with the message PermissionError: [Errno 13] Permission denied: 'samplefile.txt'. However, opening the file in read mode will not cause any issues, as we have permission to read it.

file = open("samplefile.txt", "a")

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    file = open("samplefile.txt", "a")
PermissionError: [Errno 13] Permission denied: 'samplefile.txt'

Is a directory error

We can open files using the open() function in Python. However, if you try to open a directory using the open() function, the program raises the IsADirectoryError exception.

file = open("/path/to/directory", "r")

In this code, /path/to is a directory. Hence, the program throws an IsADirectoryError exception with the message IsADirectoryError: [Errno 21] Is a directory

Traceback (most recent call last):
  File "/path/to/code.py", line 1, in <module>
    file = open("/path/to/directory", "r")
IsADirectoryError: [Errno 21] Is a directory: '/path/to/directory'

Connection error

In Python, the ConnectionError exception occurs due to network issues such as a lost internet connection, DNS errors, server downtime, or when the client fails to establish a connection to the server within a specified time limit. For instance, using the requests module to make an API call without connecting the system to the network results in the ConnectionError exception, as shown below:

import requests
response = requests.get('https://jsonplaceholder.typicode.com/todos/1')

Output:

Traceback (most recent call last):
  File "/path/to/code.py", line 4, in <module>
    response = requests.get('https://jsonplaceholder.typicode.com/todos/1')
  .
  .
  File "/home/user/.local/lib/python3.10/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='jsonplaceholder.typicode.com', port=443): Max retries exceeded with url: /todos/1 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7400d45a6c80>: Failed to resolve 'jsonplaceholder.typicode.com' ([Errno -3] Temporary failure in name resolution)"))

Connection refused error

The ConnectionRefusedError is a subclass of ConnectionError, which specifically indicates that a connection attempt was explicitly refused by the remote host. It occurs due to an incorrect IP address or port number, firewall blocking, or if the server is not running or has reached its maximum capacity for pending connections. For example, consider the following code:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("localhost", 9999))

We have not run any application on port 9999. Hence, when the Python program tries to connect to the port, the program runs into ConnectionRefusedError with the message ConnectionRefusedError: [Errno 111] Connection refused, as shown below:

Traceback (most recent call last):
  File "/path/to/code.py", line 3, in <module>
    s.connect(("localhost", 9999))
ConnectionRefusedError: [Errno 111] Connection refused

How to avoid system-level errors in Python?

System-level errors in Python are often caused by missing files, insufficient permissions, memory limitations, or network failures. Although we cannot always prevent them, we can minimize system-level errors through careful resource management, validation checks, and proper exception handling. You can use the following practices to avoid system-level errors in Python.

Always check whether files and directories exist before performing file operations.
Check access permissions to ensure the program has the required read, write, and execute permissions for files and directories.
Manage memory efficiently by avoiding extremely large data structures and using generators or batch processing for large datasets.
Use context managers (with statement) when working with files, sockets, or other resources to ensure they are automatically closed after use.
Ensure required system resources, such as disk space, memory, and network connectivity, are available.

The best way to avoid system-level errors is to anticipate potential failures and write defensive code that validates resources and handles exceptions gracefully, since these are mostly caused by environmental or resource constraints.

Logical errors in Python

A logical or semantic error in Python occurs when a program runs without crashing or raising exceptions but produces incorrect or unintended results due to flawed code logic. Unlike syntax errors or runtime errors, Python cannot detect logical errors automatically because the code is syntactically valid and executes successfully. Logical errors occur due to incorrect algorithms, wrong conditions, faulty calculations, or mistaken assumptions in program logic.

For example, consider that you are writing a Python application to determine the voting eligibility of a person based on their age. It is given that a person aged 18 or older is eligible to vote. Now consider the following code:

if age > 18:
    print("Eligible to vote")
else:
    print("Not eligible")

This code executes successfully. But it produces incorrect output for people aged 18 due to an incorrect condition. Hence, we should have used the >= operator instead of the > operator in the if block.

Logical errors can also occur due to incorrect use of logical operators. For example, suppose we need to determine whether a person is of working age, defined as 18 to 60 years old, inclusive. Now, consider the following code:

age = 70
if age >= 18 or age <= 60:
    print("Working age")

Output:

Working age

This code prints "Working age" even if the person is over 60 years old because the first condition is True, and the or operator evaluates to True if either operand is True. Hence, we should have used the and operator rather than the or operator to get the correct output.

Logical errors are much harder to detect because the Python application does not run into exceptions. The output may appear reasonable but will still be incorrect. We can detect and avoid logical errors through unit testing, debugging, and code review, ensuring there are no flaws in the code.

Know your errors before your users do

No matter how experienced you get, errors in an application never fully disappear. What changes is how quickly you recognize them, how calmly you respond, and how efficiently you solve the errors. In this article, we discussed the different syntax errors, runtime errors, system-level errors, and logical errors in Python and how to avoid them. While we cannot eliminate errors entirely, we can significantly reduce them by following error-handling best practices.

It is important to recognize that not every error can be caught during development. Certain bugs only emerge in production, often triggered by specific user actions in unpredictable sequences that may not be covered by testing. When this occurs, users may encounter an error before the development team discovers it. This is where error tracking and application monitoring become essential.

Honeybadger provides error tracking, logging, uptime monitoring, and performance monitoring under one roof. Sign up for a free trial of Honeybadger to monitor your applications by combining error tracking, logging, and uptime monitoring, so you always know the state of your application and can catch errors before they snowball.

Heroku vs AWS

2026-04-21T00:00:00+00:00

Heroku vs AWS: these cloud platforms represent fundamentally different approaches to application cloud hosting. The decision between them often determines whether your team ships features in hours or spends days configuring infrastructure. Both platforms represent different philosophies in cloud computing, with Heroku prioritizing developer experience while AWS maximizes infrastructure control. Modern applications rely on technologies called cloud computing to deliver scalable, on-demand infrastructure without maintaining physical servers.

This Heroku vs AWS comparison examines the factors that impact your choice: deployment workflows (Git-push simplicity versus infrastructure configuration), cost structures (transparent per-dyno pricing versus compounded service charges), scaling approaches (instant dyno multiplication versus auto-scaling policies), operational overhead (managed updates versus manual maintenance), and migration complexity (platform lock-in versus portable architectures).

Heroku vs AWS: things to consider

Heroku platform operates as an opinionated PaaS platform built atop AWS infrastructure, abstracting away server management and underlying infrastructure completely. The platform's containers equivalent on Heroku are called dynos, which handle application processes. It runs on lightweight Linux containers that boot from a slug (compiled application bundle). Each dyno receives 512MB of memory in the cheapest tier and scales to multiple gigabytes in performance tiers. The filesystem is ephemeral, which resets with each dyno restart. This forces stateless application design and external storage for persistent data.

As an AWS IaaS platform, AWS equips developers with discrete building blocks like EC2 instances, Lambda functions, ECS containers, RDS databases, Amazon Simple Storage Service (or Amazon S3) buckets, that developers assemble into custom architectures and complex infrastructure. An EC2 t3.micro instance provides 1GB of memory and two vCPUs with full filesystem access and persistent storage via EBS volumes. Lambda executes functions in response to events with 128MB to 10GB configurable memory. AWS functions as a comprehensive cloud service provider offering over 200 services, while Heroku operates as a focused PaaS layer.

This enables architectures ranging from monolithic EC2 deployments to serverless microservices, but shifts infrastructure decisions to development teams. AWS Elastic Compute Cloud (EC2) provides the foundational infrastructure that many cloud services build upon, including Heroku itself.

Heroku's infrastructure buildpack system detects application programming languages and automatically configures runtime environments. A Node.js application with a package.json triggers the Node.js buildpack, which installs dependencies, compiles assets, and prepares the execution environment. AWS offers similar automation through Elastic Beanstalk, but most DevOps engineers (AWS developers) construct custom deployment pipelines using CloudFormation templates or Terraform configurations that explicitly define every resource.

The networking models differ substantially. Heroku applications receive an *.herokuapp.com subdomain by default, with routing managed transparently through the platform's load balancer. Custom domains require DNS configuration, but no manual load balancer setup.

AWS, on the other hand, requires explicit virtual private cloud (VPC) configuration, subnet allocation, security settings through security group rules, elastic load balancing through load balancer provisioning, and target group configuration before applications become accessible on the internet. Engineers who manage AWS infrastructure spend significant time in the AWS Management Console configuring VPCs, security groups, and access management policies.

Deployment workflows: Git-push simplicity vs infrastructure configuration

For deploying an app, Heroku centers on Git-based workflows with minimal configuration. Developers push code to a Heroku Git remote, triggering an automatic build process that creates a new slug, deploys it across dynos, and performs health checks before routing traffic. The entire sequence completes without manual intervention:

# Initial Heroku setup
heroku create myapp
heroku addons:create heroku-postgresql:essential-0

# Deploy with a single command
git push heroku main

# View application logs
heroku logs --tail

# Scale application
heroku ps:scale web=2

Heroku's intuitive Heroku platform dashboard allows developers to manage deployments, scale dynos, and monitor applications without deep cloud computing knowledge. The platform handles dependency resolution, asset compilation, and process management based on the Procfile specification, enabling rapid app development without manual server configuration. A typical Node.js application requires only a Procfile defining process types:

web: node server.js
worker: node worker.js

Heroku's dyno manager reads this configuration and manages process lifecycles. Database credentials, API keys, and configuration values pass through environment variables accessible via process.env, with the platform injecting connection strings for provisioned add-ons.

AWS services require explicit infrastructure definition and AWS environment setup before code deployment, introducing the AWS learning curve that teams must navigate. Using AWS Elastic Beanstalk provides the closest analog to Heroku's experience, but even this managed service demands configuration files defining environment settings, instance types, and scaling parameters. A minimal Elastic Beanstalk configuration in .ebextensions/nodecommands.config will look like this:

option_settings:
  aws:elasticbeanstalk:container:nodejs:
    NodeCommand: "node server.js"

  aws:autoscaling:launchconfiguration:
    InstanceType: t3.small

  aws:elasticbeanstalk:environment:
    EnvironmentType: LoadBalanced

Deployment proceeds through the AWS powerful CLI or EB CLI, which packages the application, uploads it to S3, and updates the environment:

# Initialize Elastic Beanstalk
eb init -p node.js-18 myapp

# Create environment with database
eb create myapp-prod --database.engine postgres --database.size 20

# Deploy application
eb deploy

# Access logs
eb logs

# Scale instances
eb scale 2

If you want to bypass Elastic Beanstalk, you would have to construct deployment pipelines manually. A common pattern uses AWS CodePipeline to orchestrate source control integration, CodeBuild for compilation, and ECS for container orchestration.

Container-based AWS deployments using ECS Fargate require a task definition JSON specifying container images, CPU and memory allocation, environment variables, and networking configuration:

{
  "family": "myapp",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512",
  "containerDefinitions": [
    {
      "name": "web",
      "image": "123456789.dkr.ecr.us-east-1.amazonaws.com/myapp:latest",
      "portMappings": [
        {
          "containerPort": 3000,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "DATABASE_URL",
          "value": "postgresql://user:pass@db.example.com:5432/dbname"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/myapp",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ]
}

This task definition connects to a service definition that integrates with Application Load Balancers, configures health checks, and manages deployment strategies (rolling updates, blue-green deployments). The manual assembly process provides control over resource allocation but increases deployment complexity by an order of magnitude compared to Heroku's Git-push model.

Cost models and pricing analysis

Heroku vs AWS pricing comparison may not be as straightforward as you might think. The choice between these cloud platforms ultimately depends on whether teams value deployment simplicity or infrastructure flexibility. Heroku's pricing follows a straightforward per-dyno model. The Basic plan charges $7 monthly per dyno, providing 512MB memory and automatic SSL. Standard dynos cost $25-50 monthly with 512MB-1GB memory. Performance dynos range from $250-500 monthly for 2.5GB-14GB memory. Add-ons follow similar transparent pricing. Heroku Postgres Essential-0 costs $5 monthly for 1GB storage and 20 connections, while Essential-1 provides 10GB storage for $9 monthly.

AWS pricing compounds multiple service charges. AWS instance pricing starts at approximately $15 monthly for an EC2 t3.small instance (2 vCPUs, 2GB memory) in us-east-1 with on-demand pricing, but requires additional charges for EBS storage ($0.10 per GB-month), data transfer ($0.09 per GB egress), and load balancer operation ($16 monthly base plus $0.008 per LCU-hour). While AWS offers free tiers for experimentation (750 hours of t2.micro monthly), reserved instances reduce EC2 costs by 40-60% with one or three-year commitments for production workloads, introducing capacity planning complexity we don't have in Heroku's monthly subscriptions.

Consider a Node.js/React application serving 10,000 monthly active users with moderate traffic patterns. The architecture includes a web server, background worker, PostgreSQL database, and Redis cache:

Heroku deployment:

2x Standard-1X web dynos (1GB each): $50/month
1x Standard-1X worker dyno: $25/month
Heroku Postgres Standard-0 (64GB, 120 connections): $50/month
Heroku Redis Premium-0 (100MB): $15/month
Total: $140/month

AWS deployment (Elastic Beanstalk):

2x t3.small EC2 instances: $30/month
Application Load Balancer: $20/month (estimated with LCU charges)
RDS PostgreSQL db.t3.micro (1 vCPU, 1GB, 20GB storage): $25/month
ElastiCache Redis cache.t3.micro (0.5GB): $12/month
EBS storage (20GB across instances): $2/month
Data transfer (estimated 50GB egress): $4.50/month
Total: $93.50/month

AWS deployment (ECS Fargate):

2x Fargate tasks (0.5 vCPU, 1GB each, continuous): $36/month
Application Load Balancer: $20/month
RDS PostgreSQL db.t3.micro: $25/month
ElastiCache Redis cache.t3.micro: $12/month
Data transfer: $4.50/month
Total: $97.50/month

Organizations leveraging cloud computing services from either provider must consider long-term operational costs beyond monthly subscription fees. Heroku's simplified pricing will eliminate surprise charges from misconfigured computing resources, and there will be no accidental fees. AWS requires constant cost monitoring through Cost Explorer, budget alerts, and tagging strategies to prevent billing surprises.

Note: It may look like you will spend less on AWS, but the extra technical expertise required to make AWS run as expected might increase the overall expense for deployment in the long run.

Deployment features comparison

Heroku and AWS offer overlapping capabilities with vastly different implementation approaches. The following table compares deployment features across Heroku and three common AWS deployment patterns (Elastic Beanstalk (managed PaaS), ECS Fargate (container orchestration), and direct EC2 management).

Features marked "Manual" require custom implementation through scripts, configuration management tools, or third-party services rather than automation provided by the platform.

Scaling AWS vs Heroku: performance and operational tradeoffs

Heroku and AWS manage cloud infrastructure differently, as we know. Heroku hides complexity behind automation while AWS demands explicit configuration at every level. Scaling Heroku applications works through dyno multiplication for horizontal scaling (launching multiple app instances) and dyno type selection for vertical scaling. Scaling to 10 Standard-2X dynos (creating multiple app instances of 1GB each) happens instantly via the CLI or dashboard without application code changes. The platform's intelligent routing distributes requests across available dynos using a routing algorithm that occasionally produces request queuing under heavy load. Performance-M dynos provide dedicated computing resources and reduced request latency compared to Standard dynos' shared infrastructure.

AWS auto-scaling operates at the infrastructure layer. Elastic Beanstalk configurations specify scaling triggers based on CloudWatch metrics. A typical configuration scales EC2 instances when the average CPU exceeds 70% for five minutes:

option_settings:
  aws:autoscaling:asg:
    MinSize: 2
    MaxSize: 10

  aws:autoscaling:trigger:
    MeasureName: CPUUtilization
    Statistic: Average
    Unit: Percent
    UpperThreshold: 70
    UpperBreachScaleIncrement: 2
    LowerThreshold: 30
    LowerBreachScaleIncrement: -1

ECS Fargate enables finer-grained scaling through target tracking policies on custom CloudWatch metrics. Teams emit application-level metrics (active connections, queue depth, response times) to CloudWatch and configure scaling policies responding to business logic rather than infrastructure metrics. This scaling responds to load patterns that infrastructure metrics miss.

Lambda functions scale automatically to concurrent execution limits (1,000 concurrent invocations by default, increasable via service quotas). Each invocation runs in an isolated execution context with dedicated memory and CPU allocation. Cold start latency (100-800ms for Node.js functions) impacts performance for infrequently accessed endpoints but disappears under sustained load as Lambda maintains warm execution contexts.

Heroku's ephemeral filesystem and 30-second request timeout constrain certain workload types. Long-running background jobs exceeding 30 seconds require worker dynos rather than web dynos. File uploads must stream directly to S3 or similar external storage since filesystem writes disappear on dyno restart.

AWS imposes no such timeout restrictions. EC2 and ECS tasks run indefinitely, enabling long-running processes, persistent WebSocket connections, and stateful application architectures. The flexibility introduces operational responsibilities that Heroku manages automatically. Teams must implement health checks, graceful shutdown handlers, and restart policies that Heroku provides out of the box.

Network performance follows similar patterns. Heroku's shared routing infrastructure introduces variable latency (typically 10-50ms) between request receipt and application processing. AWS Application Load Balancers add 1-5ms latency with predictable performance characteristics. Teams requiring single-digit millisecond response times deploy on AWS with dedicated load balancers, placement groups for reduced inter-instance latency, and Enhanced Networking for 25-100Gbps network bandwidth.

When to choose Heroku, when to choose AWS, and when to use both

Understanding the architectural differences between Heroku and AWS helps you select the right foundation for web apps, APIs, and mobile apps. Heroku suits teams prioritizing rapid app development velocity and a streamlined deployment process over infrastructure optimization, avoiding the AWS learning curve. Startups validating product-market fit, agencies shipping client projects rapidly, and small teams without dedicated operations staff benefit from Heroku's reduced operational overhead. The platform eliminates infrastructure decisions that distract from product development.

Choose Heroku when building highly scalable web applications that fit platform constraints: stateless web applications, API servers, scheduled background jobs with reasonable duration, and traffic patterns under 100 requests per second. The 30-second request timeout handles most HTTP endpoints adequately. Applications requiring PostgreSQL, Redis, Kafka, Elasticsearch, or other common infrastructure components benefit from the managed services Heroku provides through add-ons with automatic backups, monitoring tools, and maintenance.

AWS services become necessary when applications demand infrastructure flexibility beyond Heroku's abstractions, despite the steeper AWS learning curve for DevOps engineers. Machine learning inference workloads requiring GPU instances, real-time analytics processing terabytes daily, multi-region active-active deployments, or HIPAA/SOC 2 compliance with dedicated tenancy all exceed Heroku's capabilities. Teams optimizing infrastructure costs at scale achieve 50-70% savings through reserved instances, spot instances, and resource right-sizing, which Heroku's fixed pricing model prevents.

Hybrid architectures combine both platforms strategically. A common pattern deploys the core web application on Heroku for rapid iteration while offloading specialized workloads to AWS, like video transcoding in AWS Lambda, data analytics in EMR, machine learning inference in SageMaker, and file storage in S3, all part of the broader AWS ecosystem. Heroku applications' integration with AWS services happens through API endpoints or direct SDK calls using IAM credentials stored in Heroku config vars.

Migration considerations and common pitfalls

Migrating from Heroku to AWS requires addressing architectural assumptions baked into Heroku applications. Applications writing temporary files, caching assets locally, or storing session data on disk fail when moved to AWS without refactoring to Redis, S3, or database-backed storage.

Environment variable management becomes explicit during AWS migration. Heroku injects DATABASE_URL and other add-on credentials automatically, while AWS requires manual configuration through Systems Manager Parameter Store, Secrets Manager, or environment variables in task definitions. The migration checklist must enumerate every config var and establish AWS equivalents:

# Export Heroku environment variables
heroku config --shell > .env.heroku

# Convert to AWS Systems Manager parameters
while IFS='=' read -r key value; do
  aws ssm put-parameter \
    --name "/myapp/prod/$key" \
    --value "$value" \
    --type "SecureString"
done < .env.heroku

Database migration requires careful planning around downtime windows and replication lag. The recommended approach establishes AWS RDS as a follower to Heroku using logical replication, allowing traffic cutover with minimal downtime.

Once replication lag stabilizes (typically under one second), maintenance mode redirects traffic to AWS while the final database sync completes. The process works for databases under 100GB, but becomes impractical for terabyte-scale datasets requiring offline migration windows or more sophisticated replication tooling.

Applications using Heroku add-ons, including review apps for testing branches, must replace each with AWS equivalents or third-party SaaS. Heroku Redis maps to ElastiCache, Heroku Kafka to Amazon MSK, Heroku Scheduler to CloudWatch Events + Lambda.

DNS cutover represents the final migration step. Heroku applications respond to myapp.herokuapp.com and custom domains through Heroku's edge routing. Amazon Web Services migrations require updating DNS records to point at CloudFront distributions, Application Load Balancers, or API Gateway endpoints.

Choosing the right platform for your team

This Heroku vs AWS comparison makes one thing clear: these cloud platforms aren't competitors—they're cloud services built for different stages of a product's life, each with distinct approaches to cloud computing and infrastructure management. Heroku gives you the fastest path to a working, scalable application with almost no operational overhead, while Amazon Web Services offers the flexibility needed for complex architectures, large-scale optimizations, and workloads that exceed platform limits.

DevOps engineers moving between the two must plan for changes in deployment workflows, environment management, and infrastructure responsibilities, especially when shedding Heroku’s built-in conveniences. The comparison between Heroku and AWS extends beyond pricing to include developer productivity, operational overhead, and long-term scalability.

Like this article? Join the Honeybadger newsletter to learn more.

Why every web developer should explore machine learning

2020-03-18T00:00:00+00:00

I don't have kids yet, but when I do, I want them to learn two things:

Personal finance
Machine learning

Whether or not you believe that the singularity is near, there's no denying that the world runs on data. Understanding how that data is transformed into knowledge is critical for anyone coming of age these days – and even more so for developers.

This is the first article in a series that will attempt to make machine learning (ML) accessible to full-stack Ruby developers. By understanding the ML tools at your disposal, you'll be able to help your stakeholders make better decisions. Future articles will focus on individual techniques and practical examples, but in this one, we're setting the stage – showing you a map and placing a pin that says "you are here."

Humble Beginnings

Artificial intelligence (AI) and machine learning are nothing new. Back in the 1950s, Arthur Samuel built a computer program that could play checkers. He used "alpha-beta pruning" – a common search algorithm.

The 1960s saw the advent of multi-layered neural networks and the nearest-neighbor algorithm, which is used to find optimal pathing in warehouses.

So if AI is so old, why are AI startups so trendy? In my opinion, there are two reasons for this:

Computing power (see Moore's Law)
The amount of data being added to the internet every day

There are two statistics related to the amount of data being created on a daily basis that blow my mind every time I think about them:

As of 2018, we are producing 2.5 quintillion bytes of data every day. No doubt this number has only increased since this Forbes article was published.
Over the last two years alone, 90% of the data in the world was generated.

Together, what this means is that (1) the hardware necessary to store data and run algorithms continues to become more affordable and (2) the amount of data available to train the ML models is increasing at a crazy-fast pace.

Every day we are interacting with, being influenced by, and contributing to the world of artificial intelligence and machine learning. For example, you can thank (or blame) algorithms for the following:

Your credit line
Helping diagnose illness
Maybe even whether or not you got the job
Helping you find the most efficient route given current traffic conditions
Alexa understanding exactly what you mean when you tell her you just sneezed
Spotify introducing you to your new favorite song

The reason I brought up my hypothetical future kids is this: I want them to understand how their digital lives influence their "real" lives, the implications of their data privacy decisions, and how to form their own opinions of when they should trust the machine vs. when they shouldn't.

In the remainder of this post, I want to give an overview of the three kinds of machine learning that I've studied: supervised learning, unsupervised learning, and reinforcement learning. We'll talk about what makes each approach unique and the problems each is especially good at solving.

Supervised Learning

Supervised learning is... well, supervised by humans. :) Imagine that we're building a supervised learning system to decide who gets approved for a mortgage. Here's how it might work:

The bank compiles a dataset that maps customer attributes (age, salary, etc.) to outcomes (repayment, default, etc.).
We train our system using the data.
The system uses what it learns to guess future outcomes based on an applicant's attributes.
If the algorithm guesses correctly, we tell it, "Great job! You're right." But if it's wrong, we tell it, "No, you're incorrect. Please improve and try again."

This example is considered a "classification" problem because the output of the algorithm is a category – in this case, approved or not approved. Other examples of classification problems include deciding whether or not:

a person has an illness
an x-ray has a broken bone
an e-mail is spam

If you're curious to learn more about the math behind ML classification algorithms, Google any of these: naive Bayes classifiers, support vector machines, logistic regression, neural networks, random forests.

In addition to classification problems that render a "yes/no" outcome, supervised learning can also be utilized to solve regression problems – here, we make a prediction on a continuous scale, for example:

the future value of a stock
the probability that the New England Patriots win the Super Bowl
the average salary a company needs to offer a candidate for them to accept

Examples of algorithms used for supervised regression problems include linear regression, non-linear regression, and Bayesian linear regression.

Unsupervised Learning

With our supervised learning example, we predefined the classification categories. The mortgage applicant was either approved or denied.

With unsupervised learning, we don't provide the categories. They're not available to us. The algorithm must come up with its own conclusions.

Why would we want to use an unsupervised approach?

Sometimes we don't know categories beforehand. Much of the data floating around the internet is unstructured – i.e., lacking labels.
Other times, we don't know what we're looking for, so we can ask the algorithm to find patterns/features that can be useful for categorization.

Another way to handle unstructured data with machine learning is to simply have humans look at it and manually label it. There are lots of companies that hire workers to manually classify data: labeling data.

Approaches to Unsupervised Learning

Two techniques that are commonly used in unsupervised learning are association and clustering.

Association: Imagine that you're Amazon. You have a lot of customer data, purchase history, etc. By using unsupervised learning, you could partition customers into "types of shoppers" – maybe figuring out that those who buy pink umbrellas are more likely to also purchase Matcha tea.

Clustering: Clustering looks at your data and partitions it into a specified number of groups, or clusters. For example, maybe you have a bunch of housing data and you want to see if there are any features (possibly crime data?) that can predict what neighborhood the home is in. Or, techniques like cosine similarity can be used for text classification (e.g., is this article about tennis, cooking, or space?).

If you're interested in learning more about specific unsupervised learning techniques, Google search k-means clustering, cosine similarity, hierarchical clustering, and k-nearest-neighbor clustering.

Reinforcement Learning

This subset of machine learning is commonly used in games because it utilizes goal-oriented algorithms. Unlike supervised learning, each decision is NOT independent – given a current input, the algorithm makes a decision and the next input depends on this decision.

Just like I give my dog a pat on the head when he stops incessantly barking when the doorbell rings or put him in his kennel when he won't shut up, reinforcement algorithms are rewarded when making a goal-optimizing decision (e.g., scoring the max number of points) and penalized when making a poor one.

The obvious applications for reinforcement learning algorithms include:

games like chess and Go (I highly recommend the AlphaGo documentary on Netflix, if you haven't seen it already.)
robotics (teaching bots to complete desired tasks)
autonomous vehicles
robo-advisors that are trained to beat the stock market

If you're interested in learning more about the algorithms behind reinforcement learning, Google search Q-learning, state-action-reward-state-action (SARSA), DQN, and the asynchronous advantage actor critic.

Remember to learn more

I hope that these examples have helped you gain a grasp on machine learning techniques and how each is used to influence the crazy world we live in today. While I sometimes find myself overwhelmed with all that there is to learn, starting somewhere is better than doing nothing, and remember that much of this is actually not very new at all – we are just hearing about it more as data becomes more available and processing power cheaper.

Cross-cluster associations in Rails

2023-01-30T00:00:00+00:00

One of the beauties of the Rails framework is the ability to utilize Ruby on Rails associations in your models. These associations allow you to access collections of records in your code with pleasant syntax, abstracting away the need to write underlying SQL queries. That abstraction holds as long as all your data lives in one place. The moment your tables are spread across separate database clusters, certain association types stop working.

This article walks through exactly where that boundary is and what Rails provides to work within it. We start with why the problem occurs and which associations in Rails are affected, and move into the database configuration and model hierarchy that supports multiple clusters and many to many relationships. From there we'll cover how different different data access patterns each interact with that decomposition.

If you're looking for a Rails associations tutorial that covers multi-database setups specifically, this is it. We will also discuss many other things, so stick around.

Why databases end up in different clusters

When a Rails application stores all its data in a single database, Active Record associations are handled transparently, and you never think about the underlying SQL. The moment your data lives across multiple database clusters, that transparency breaks down. A JOIN requires both tables to exist in the same database server. Attempting one across clusters produces an ActiveRecord::StatementInvalid error like this:

ActiveRecord::StatementInvalid (Table 'people_cluster.humans' doesn't exist)

This isn't a configuration mistake. It's a hard physical constraint: database servers cannot JOIN against tables they don't host. We have this problem in has_many :through and has_one :through associations, because those are the association types that generate intermediate JOIN queries. Direct has_many or belongs_to relationships don't require a join so they work across clusters without any modification.

Understanding when you'll hit this boundary is the first step. If a User lives in the accounts database and a Post lives in the content database, User has_many :posts works fine. But if you add an intermediate Subscription model in the billing database and define User has_many :posts, through: :subscriptions, Rails will attempt to join subscriptions and posts in a single query. That's where the cluster boundary becomes a problem.

The three-tier database configuration

Before writing any model code, the database configuration needs to reflect the multi-cluster layout. Rails uses a three-tier structure in config/database.yml for this purpose. Each top-level environment key contains nested database names, and each of those contains the connection details for that cluster.

# config/database.yml
default: &default
  adapter: postgresql
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

development:
  primary:
    <<: *default
    database: myapp_primary_dev

  accounts:
    <<: *default
    database: myapp_accounts_dev
    migrations_paths: db/accounts_migrate

  content:
    <<: *default
    database: myapp_content_dev
    migrations_paths: db/content_migrate

production:
  primary:
    <<: *default
    database: myapp_primary_prod
    username: <%= ENV['DB_USER'] %>
    password: <%= ENV['DB_PASSWORD'] %>

  accounts:
    <<: *default
    database: myapp_accounts_prod
    username: <%= ENV['DB_USER'] %>
    password: <%= ENV['DB_PASSWORD'] %>

  content:
    <<: *default
    database: myapp_content_prod
    username: <%= ENV['DB_USER'] %>
    password: <%= ENV['DB_PASSWORD'] %>

The migrations_paths key is non-optional if you want Rails generators and db:migrate to route migrations to the correct directory. Without it, all migrations default to db/migrate and get applied to the primary database. Each secondary database should also have a corresponding abstract record class that Rails models inherit from. Generators handle this automatically when you pass the --database flag:

rails generate model Subscription plan:string --database accounts

This produces an AccountsRecord class if one doesn't already exist, and the generated Subscription model inherits from it.

Abstract record classes and connection routing

The abstract record classes are the mechanism Rails uses to route queries to the correct cluster. Each one calls connects_to to declare which database it maps to for writing and reading operations. Your application will typically have three layers in this hierarchy.

# app/models/application_record.rb
class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true

  connects_to database: { writing: :primary, reading: :primary }
end

# app/models/accounts_record.rb
class AccountsRecord < ApplicationRecord
  self.abstract_class = true

  connects_to database: { writing: :accounts, reading: :accounts }
end

# app/models/content_record.rb
class ContentRecord < ApplicationRecord
  self.abstract_class = true

  connects_to database: { writing: :content, reading: :content }
end

The User model is a good anchor for understanding this hierarchy. It lives in the accounts cluster and inherits from AccountsRecord. Models in the content cluster inherit from ContentRecord. Everything else inherits from ApplicationRecord and hits the primary database. This inheritance chain is how Active Record determines which connection pool to use when executing a query. It walks up the class hierarchy until it finds a class that has called connects_to.

A common mistake is calling establish_connection on individual models instead of using abstract classes. Each establish_connection call opens a separate connection pool. If you have 50 models in the accounts database, each calling establish_connection, you end up with 50 connection pools pointing at the same server. Abstract classes solve this by sharing a single pool across all models that inherit from them.

How cross-cluster associations in Rails actually work

The disable_joins: true option is the direct mechanism for making through associations work when the involved tables live in different clusters. Rails has_many is the most commonly used association type, and it's the one most directly affected by cluster boundaries. When Rails encounters this option on an association, it abandons the single JOIN query strategy and instead issues two (or more) sequential SELECT statements, piping the IDs from the first query into a WHERE ... IN (...) clause in the second.

Here's a concrete model setup spanning three clusters. The model setup below is a many-to-many relationship, a User connects to Posts through Subscriptions, and it's the pattern that most directly exposes the cross-cluster problem.

# app/models/user.rb - lives in the accounts database
class User < AccountsRecord
  has_many :subscriptions
  has_many :posts, through: :subscriptions, disable_joins: true
end

# app/models/subscription.rb - lives in the accounts database
class Subscription < AccountsRecord
  belongs_to :user
  has_many :posts
end

# app/models/post.rb - lives in the content database
class Post < ContentRecord
  belongs_to :subscription
end

When you call user.posts, Rails generates this pair of queries instead of a single JOIN:

-- Query 1: fetch subscription IDs from the accounts cluster
SELECT "subscriptions"."id"
FROM "subscriptions"
WHERE "subscriptions"."user_id" = 1

-- Query 2: fetch posts from the content cluster using those IDs
SELECT "posts".*
FROM "posts"
WHERE "posts"."subscription_id" IN (4, 7, 12)

The first query runs against the accounts database to collect a primary key. The second runs against content. Rails resolves the relationship by following the foreign keys, user_id on subscriptions and subscription_id on posts, across the two clusters. The first query collects the primary key values from subscriptions, then passes them into the IN clause of the second query. Neither query attempts a cross-cluster join. Rails assembles the final result set in application memory. The same option works identically on has_one :through:

# app/models/user.rb
class User < AccountsRecord
  has_one :profile
  has_one :avatar, through: :profile, disable_joins: true
end

# app/models/profile.rb - accounts database
class Profile < AccountsRecord
  belongs_to :user
  has_one :avatar
end

# app/models/avatar.rb - content database
class Avatar < ContentRecord
  belongs_to :profile
end

user.avatar will execute two queries: one to get the profile_id, another to fetch the avatar record from the content cluster.

When `disable_joins` must be set explicitly

Rails does not automatically detect cluster boundaries and insert disable_joins for you. Association loading in Active Record is lazy. The SQL for an association is determined at the point the association is defined on the model, not when it's actually triggered. By the time user.posts executes, Rails has already decided whether to use a JOIN or separate queries based on the association declaration.

This means every through association that crosses a cluster boundary needs disable_joins: true on the declaration.

A practical way to audit your models is to look for any through: association where the source model and the target model inherit from different abstract record classes. If User < AccountsRecord and Post < ContentRecord, then has_many :posts, through: :subscriptions needs disable_joins: true regardless of where Subscription lives.

Eager loading across clusters

The disable_joins option affects how associations are loaded, but it does not change how eager loading strategies interact with cross-cluster data. Understanding this distinction matters for avoiding N+1 queries in multi-database setups.

eager_load is off the table for cross-cluster associations. It generates a LEFT OUTER JOIN, which has the same physical limitation as a regular JOIN , both tables must be on the same server. If you attempt User.eager_load(:posts) where posts live in a different cluster, you will get the same StatementInvalid error.

preload is the correct strategy. It issues separate queries for each association and assembles the relationship in Ruby. This is structurally identical to what disable_joins does for a single record. The difference is scale: preload batches the second query across all loaded parent records.

# This works across clusters.
# Query 1: SELECT "users".* FROM "users"
# Query 2: SELECT "posts".* FROM "posts" WHERE "posts"."subscription_id" IN (...)
users = User.preload(:posts).all

users.each do |user|
  user.posts.each { |post| puts post.title }  # No additional queries fired
end

includes will work in cases where it delegates to preload internally, which it does by default when there are no conditions referencing the associated table. If you add a .where clause that touches the associated table's columns, includes switches to eager_load behavior, and will fail across clusters. When in doubt about which strategy includes will choose, be explicit and use preload directly.

# includes delegates to preload here, works across clusters
User.includes(:posts).all

# includes switches to eager_load because of the where clause, fails across clusters
User.includes(:posts).where("posts.published = ?", true)

# Use preload + a separate where for cross-cluster filtering
User.preload(:posts).all.select { |u| u.posts.any?(&:published?) }
# Or filter in application code after loading

Scoped associations and cross-cluster filtering

One of the more subtle interactions in multi-database setups is scoped associations. When you define a scope on a has_many that crosses clusters, the scope's SQL runs against the target database, not the source.

class User < AccountsRecord
  has_many :subscriptions
  has_many :published_posts,
           -> { where(published: true) },
           through: :subscriptions,
           source: :posts,
           class_name: "Post",
           disable_joins: true
end

The where(published: true) clause gets appended to the second query, the one that runs against the content database. This is correct behavior, and it means your scopes can reference columns on the target table without issue. What you cannot do is reference columns from the intermediate table in that scope, because the intermediate query has already completed by the time the scoped query executes.

# This will fail because subscriptions.active is not a column in the content database
has_many :active_posts,
         -> { where("subscriptions.active = ?", true) },
         through: :subscriptions,
         source: :posts,
         disable_joins: true

Filter intermediate records by adding a scope to the intermediate association instead:

class User < AccountsRecord
  has_many :active_subscriptions, -> { where(active: true) }, class_name: "Subscription"
  has_many :active_posts, through: :active_subscriptions, source: :posts, disable_joins: true
end

Now the filtering on subscriptions.active happens in the first query, against the accounts database, and only the IDs from active subscriptions get passed to the second query.

Horizontal sharding and cross-shard associations

Splitting one logical database across multiple servers based on a partition key like tenant_id introduces a second dimension to the cross-cluster problem. The disable_joins mechanism still applies, but the connection routing becomes more involved.

Rails provides connected_to for switching between shards within a request:

ActiveRecord::Base.connected_to(role: :writing, shard: :shard_one) do
  User.find(1)  # Hits shard_one
end

When associations span both clusters and shards, you need to ensure both the shard context and the disable_joins option are in place. A User on shard_one accessing posts that live in a separate content database still needs the same two-query decomposition.

Rails 8 added introspection methods that make reasoning about shard topology easier at runtime:

class ShardedBase < ActiveRecord::Base
  self.abstract_class = true

  connects_to shards: {
    shard_one: { writing: :shard_one },
    shard_two: { writing: :shard_two }
  }
end

class User < ShardedBase; end

User.shard_keys         # => [:shard_one, :shard_two]
User.sharded?           # => true

ShardedBase.connected_to_all_shards do
  User.current_shard    # Yields :shard_one, then :shard_two
end

connected_to_all_shards is particularly useful for background jobs that need to process records across every shard. It iterates over each shard in sequence, switching the connection context for each block execution.

For tenant-based sharding, the lock: true default on shard switching prevents accidental tenant hopping mid-request. This is a safety mechanism: once a request is routed to a tenant's shard, the application code cannot switch to a different tenant's shard without explicitly passing lock: false. Cross-cluster associations within a single tenant's shard still use disable_joins for associations that touch a different database cluster.

Testing cross-cluster associations

Testing multi-database setups requires that your test environment mirrors the production database topology. Rails' test framework supports this, but the configuration must be explicit.

Each database in database.yml needs a test environment block. Fixtures and factory-based test data must target the correct database. If a User factory creates a record in the accounts database and a Post factory creates one in content, the association between them only works if both records exist in their respective databases within the same test transaction.

Rails wraps each test in a transaction by default, but that transaction is per-connection. With multiple databases, each connection gets its own transaction. This means test cleanup (the automatic rollback at the end of each test) happens independently on each database. If your test writes a User to accounts and a Post to content, both will be rolled back, but only if the test framework knows about both connections.

The fixtures declaration handles this automatically when models inherit from the correct abstract class. For factory-based setups (FactoryBot, Fabricator), ensure each factory's create strategy hits the right database by letting the model's own connects_to routing do the work.

# spec/factories/users.rb
FactoryBot.define do
  factory :user do
    # User inherits from AccountsRecord and writes to accounts DB automatically
    name { Faker::Name.name }
  end
end

# spec/factories/posts.rb
FactoryBot.define do
  factory :post do
    # Post inherits from ContentRecord and writes to content DB automatically
    association :subscription
    title { Faker::Lorem.sentence }
  end
end

To verify that cross-cluster associations are firing the expected number of queries, subscribe to the sql.active_record notification:

# spec/support/query_counter.rb
module QueryCounter
  def assert_query_count(expected, &block)
    count = 0
    callback = ->(_name, _start, _finish, _id, payload) do
      count += 1 unless payload[:name] == "SCHEMA" || payload[:sql].start_with?("EXPLAIN")
    end

    ActiveSupport::Notifications.subscribed(callback, "sql.active_record", &block)
    assert_equal expected, count, "Expected #{expected} queries, got #{count}"
  end
end

A has_many :through with disable_joins: true on a single record should produce exactly 2 queries. If you see 1, the join is still being attempted (and will fail in production against separate servers). If you see N+1, eager loading isn't working as expected.

Some caveats

disable_joins solves the association loading problem, but it does not extend to query chaining. You cannot chain .where, .order, or .group clauses that reference columns across clusters on a single Active Record relation:

# This does not work, you cannot filter products by order columns across clusters
customer.purchased_products.where("orders.total > ?", 100)

For queries that need to filter or sort based on data in multiple clusters, decompose them manually. Fetch the IDs or values you need from one cluster, then use them as inputs to a query against the other:

high_value_order_ids = Order.where(customer_id: customer.id)
                            .where("total > ?", 100)
                            .pluck(:id)

line_item_product_ids = LineItem.where(order_id: high_value_order_ids).pluck(:product_id)

products = Product.where(id: line_item_product_ids)

This is the same decomposition that disable_joins performs internally, but done explicitly so you can apply filtering at each stage. It's more verbose, but it makes the cluster boundaries visible in the code rather than hiding them behind associations in Rails syntax.

Heroku vs Vercel: Choosing the right platform for your app

2026-03-24T00:00:00+00:00

When it comes to deploying web applications, developers face issues picking from the huge list of cloud platform options. Two popular choices that frequently come up in discussions are Heroku and Vercel, each offering distinct approaches to application hosting and deployment. Understanding the differences between these platforms is crucial for making an informed decision that aligns with your project's needs, budget, and technical requirements.

This Heroku vs Vercel comparison examines the factors that impact your choice: architectural paradigms, deployment workflows, primary use cases, cost structures, and security models.

Heroku vs Vercel: Things to consider

Before diving into the specifics, it's important to understand the fundamental positioning of these platforms. Heroku, acquired by Salesforce in 2010, was one of the original Platform as a Service (PaaS) providers, designed to simplify application deployment across a wide range of technologies and frameworks. Vercel, on the other hand, emerged from the modern frontend ecosystem, initially created by the team behind Next.js to provide an optimal deployment experience for JavaScript and TypeScript applications.

When evaluating these platforms, consider your application's architecture, your team's expertise, scalability requirements, and budget constraints. Heroku excels at supporting traditional server-based applications and offers extensive support for multiple programming languages, including Ruby, Python, Java, PHP, Node.js, and Go. Vercel has carved out a strong niche in the frontend and full-stack app ecosystem, with exceptional support for Next.js, React, Vue, and other frontend frameworks.

The choice between them often comes down to whether you're building a traditional backend-heavy application with potentially complex server requirements, or a modern frontend-first application with serverless API routes. Your team's familiarity with the deployment paradigm also matters. Heroku follows a more traditional server-based model, while Vercel embraces serverless computing and edge computing principles.

Selling point

Heroku’s core selling point is operational simplicity for backend-heavy applications. It abstracts servers, scaling, and infrastructure into a mature PaaS with first-class support for long-running processes, background workers, scheduled jobs, and managed add-ons (Postgres, Redis, etc). If you are building traditional web apps, APIs, or worker-based systems, Heroku lets you focus on application logic and backend logic instead of platform mechanics.

Vercel's main selling point is frontend velocity and deployment confidence. It is tightly integrated with modern frameworks (especially Next.js) and offers instant global deployments, edge rendering, and preview deployments for every pull request (even though Heroku has a similar feature, review apps). Vercel is optimized for frontend apps and provides serverless infrastructure that scales automatically. For engineers prioritizing frontend performance, DX, and rapid feedback loops, look no further than Vercel.

Architectural differences between Vercel vs Heroku

The architectural philosophies of Heroku and Vercel diverge significantly, which shows their different strengths and target audiences.

Heroku operates on a dyno-based architecture. Dynos are lightweight Linux containers that run your application code. When you deploy to Heroku, your application runs on one or more dynos, which can be scaled horizontally by adding more dynos or vertically by upgrading to more powerful dyno types. This model is conceptually similar to running your application on traditional servers, but with the abstraction and convenience of a managed platform. Heroku also provides a robust add-on ecosystem, allowing you to attach databases, caching layers, monitoring tools, and other services to your application with minimal configuration.

Vercel, in contrast, embraces a serverless architecture with a strong emphasis on edge computing. When you deploy a frontend application to Vercel, your static assets are distributed across a global Content Delivery Network (CDN) for fast load times, regardless of user location. For dynamic functionality, Vercel uses serverless functions that execute on demand without requiring you to manage any server infrastructure. This serverless computing model is particularly efficient for full stack apps that combine static websites with API endpoints.

Deployment workflows

The deployment experience differs substantially between these platforms, though both aim to make deployment as straightforward as possible.

Heroku pioneered the Git-based deployment workflow. To deploy to Heroku, you typically connect your Git repository to Heroku and push your code using git push heroku main. Heroku automatically detects your application's language using buildpacks, installs dependencies, and starts your application. This workflow is intuitive for developers already familiar with Git and provides a clear mental model of deployment as a simple code push. Heroku also supports continuous deployment from GitHub repositories, automatically deploying when changes are pushed to specific branches.

The Heroku platform provides a Procfile mechanism for defining how your application should run.

Vercel takes deployment automation even further, with particularly seamless integration for frameworks it supports natively. When you connect a Git repository to Vercel, every push triggers an automatic deployment with a unique preview URL. This makes it incredibly easy to review changes before they go to production. Merging to your main branch automatically deploys to your production environment. Vercel's build system automatically detects your framework and applies appropriate build configurations, often requiring zero configuration for popular frontend frameworks like Next.js, Create React App, or Nuxt.js.

One of Vercel's standout features is its preview deployments. Every pull request gets its own deployment URL, complete with production-like infrastructure, making code reviews more effective and allowing stakeholders to interact with changes before they're merged. Vercel also makes it easy to attach custom domains to both production and preview deployments with automatic SSL.

Heroku vs Vercel: Their primary use cases

Understanding where each platform excels helps clarify which might be the better choice for your specific needs.

Heroku shines for traditional web applications, particularly those built with backend frameworks like Ruby on Rails, Django, Express.js, or Spring Boot. If you're building a monolithic application, an API server with complex business logic, or an application requiring persistent connections like real-time chat servers, Heroku provides the infrastructure and flexibility you need. The platform is also well-suited for applications requiring background job processing, scheduled tasks, or applications that need to integrate with specific databases or services through Heroku's extensive add-on marketplace.

Development teams working with polyglot architectures appreciate Heroku's multi-language support. This multi-language support makes Heroku attractive for teams running complex backends across different stacks. You can run different components of your application using different technologies, all within the same platform ecosystem. The add-on ecosystem is particularly valuable for teams that want managed services for databases, caching, monitoring, logging, and other infrastructure concerns without managing these services independently.

Vercel is the natural choice for modern frontend applications and full-stack JavaScript applications, particularly those built with Next.js. If you're building a static site, a Jamstack application, a React-based dashboard, or a marketing website with dynamic elements, Vercel's architecture delivers exceptional performance and developer experience. The platform's serverless functions are well-suited for API routes that support frontend apps, particularly for use cases like form submissions, authentication endpoints, or data transformations.

The combination of edge caching, automatic image optimization, and serverless functions makes Vercel particularly effective for content-heavy websites, e-commerce storefronts, and applications where global performance matters.

Security features in Heroku and Vercel

Security is a core concern for both platforms, but their approaches reflect different architectures. Heroku focuses on platform and workload isolation: applications run in isolated containers, HTTPS is supported via managed SSL certificates, and the platform includes network-level protections like DDoS mitigation (minimum protection). Heroku also emphasizes compliance and enterprise controls, with certifications such as SOC 2, ISO 27001, and PCI DSS. These guarantees extend to managed databases like Heroku Postgres, which inherit platform-level security controls by default.

Vercel, by contrast, follows a secure-by-default, serverless model. All deployments are automatically served over HTTPS, protected by a global edge network with built-in DDoS protection. Secrets are encrypted and scoped per environment (production, preview, development), preview deployments can be access-restricted, and enterprise plans add SSO, IP allowlisting, and deployment protection. Vercel maintains SOC 2 Type II compliance and provides fine-grained controls suited to frontend-centric and serverless applications.

Cost considerations

Understanding the cost structure of each platform is relevant for budget planning. Let's see the costs using an example of a full-stack application. Historically, Heroku offered free dynos, but today Vercel’s free tier is the more common entry point for developers experimenting or running small projects. So, for the sake of this example, we will be ignoring the free tier side of things because only Vercel offers some form of free tier.

Consider a full-stack application with a Next.js frontend with API routes, a PostgreSQL database, and Redis for caching.

Hosting

Heroku: $25 (1× Standard-1X dyno, 512 MB)
Vercel: $20 (1× Pro seat)

Postgres DB

Heroku: $50 (Standard-0, 64 GB storage)
Vercel: ≈$7 (20 GB at $0.35/GB) + compute (~$0–50)

Redis cache

Heroku: $60 (Premium-2, 250 MB)
Vercel: $0–$10 (Upstash free 256 MB tier or $10 plan)

Bandwidth

Heroku: ~Free (up to 2 TB soft limit)
Vercel: 1 TB included (then $0.15/GB)

Total (est.)

Heroku: ≈$135
Vercel: ≈$30–40

The Heroku total (~$135) assumes one dyno ($25), a Standard-0 Postgres ($50), and Redis Premium-2 ($60). Vercel’s total (~$30–40) is roughly $20 for Pro plus minimal DB/cache: Neon storage ($7) and possibly an Upstash plan ($5–10). Actual Neon compute usage (serverless scaling) could raise Vercel’s cost if the database is very active; likewise, as the app grows, heavy bandwidth or function invocation beyond free allowances could add fees.

If you take advantage of the free plan provided by Vercel, you may notice significant savings.

While the estimated Vercel total appears lower, it is important to understand how Vercel pricing scales. Unlike Heroku’s largely fixed, tier-based pricing, Vercel follows a usage-based model. The Pro plan includes a base monthly fee, but many resources are metered and billed linearly as usage increases.

Choosing the right platform for your team

Selecting between Heroku vs Vercel isn't about finding the objectively "better" platform; it's about identifying which one aligns with your specific needs. Heroku excels for traditional backend applications, supports multiple programming languages, has background jobs, and teams want an all-in-one cloud platform with a mature ecosystem. Vercel shines for modern frontend apps, static sites, Next.js projects, and teams prioritizing global performance and the seamless developer experience of automatic preview deployments.

The most pragmatic approach is to evaluate both platforms against your actual requirements. Consider your technology stack, team expertise, scalability needs, and budget. Many teams even pilot projects on each platform to experience the workflows firsthand. Remember that your choice may not be permanent. Some people reconsider their platform choice to reduce vendor lock-in, and both Heroku and Vercel make it possible to migrate. Many successful organizations often start with one platform and evolve their strategy as needs change. The platform that reduces friction, supports your stack, and enables you to deliver value quickly is the right choice for your team.

Whichever platform you choose, you'll want visibility into what's actually happening in production. Honeybadger works with both—full-stack monitoring that takes minutes to set up. Try it free for 30 days.

Building a Django Chat App with WebSockets

2022-09-22T00:00:00+00:00

Django is well known for being used to develop servers for HTTP connections and requests for applications. Unfortunately, when building Django chat app or any chat app that requires the connection to remain open for a two-way connection, using an HTTP connection is inefficient.

WebSockets provide a means of opening a two-way connection between the client and the server so that all users connected to the open network can get related data in real time. It is a stateful protocol, which means connection authentication is only required once; the client credential is stored, and there is no further need for authentication until the connection is lost.

In this article, I’ll briefly introduce you to WebSocket and its usefulness. Then, I’ll show you how to use it in Django using Django channels and create WebSocket connections with JavaScript to connect with the Django server.

We will build a simple chatbox to make things more realistic. You can find the code on GitHub.

Prerequisites

Basic understanding of Django.
Basic understanding of JavaScript.

Characteristics of WebSockets

WebSockets is a bidirectional protocol. Therefore, the client and server can exchange data simultaneously without delays or intervention. WebSockets is considered full-duplex communication for the same reason.
WebSockets is a stateful protocol. Therefore, after initial connection authentication, the client credential is saved, and further authentication is not required until the connection is lost.
WebSockets don't need any special browsers to function; it works on all browsers.

When to use WebSockets

WebSockets are used when you want to build any kind of real-time chat application, ranging from complex applications, such as multiplayer games played on the internet, to less complex ones, such as chat applications.

An alternative method of building a chat app without using WebSockets is to use JavaScript to query the database after a few seconds to get current data from the chatbox. For example, you could add a database connection to store messages in a chat log. As you can imagine, this is not scalable because if there are thousands of users, the number of requests it could generate might cause the server to crash. Additionally, this method will not work when you want to build something like a video call application.

How to use Django WebSockets

Using WebSockets in Django utilizes asynchronous Python and Django channels, making the process straightforward. Using Django channels, you can create an ASGI server, and then create a group where users can send text messages to all the other users in the group in real time. This way, you are not communicating with a particular user, but with a group, multiple users can be added.

If you are chatting with your friend on Twitter, you and your friend are in one group, represented by the chatbox.

Configure Django to use ASGI

If you don’t already have a Django project, create a folder where you want to store the code for your project, cd into it, and run startproject to create a new Django project:

django-admin startproject project .

Now, create a new Django chat app by running $ python3 manage.py startapp app.

You need to inform your Django project that a new app has been added. To do this, update the project/settings.py file and add 'app' to the INSTALLED_APPS list. It’ll look like this:

# project/settings.py
INSTALLED_APPS = [
   ...
   'chat',
]

Now install Django channels by running the following command on your command line.

pip install channels

Update the project/settings.py file and add 'channels' to the INSTALLED_APPS list:

# project/settings.py
INSTALLED_APPS = [
   ...
   'channels',
]

While you are in the settings.py file, you need to set a configuration to enable the Django channel and Django to communicate with each other using a message broker. We can use a tool like Redis for this, but for this tutorial, we will use the local backend. Paste the following code into your settings.py file:

ASGI_APPLICATION = "project.routing.application" #routing.py will be created later
CHANNEL_LAYERS = {
    'default': {
        'BACKEND': "channels.layers.InMemoryChannelLayer"
        }
    }

In the code above, ASGI_APPLICATION is needed to run the ASGI server and tell Django what to do when an event happens. This configuration will be placed in a file named routing.py.

Build a minimalistic Django chat app

Next, we will create a chat app chatbox that authenticated users can access via a URL and chat with each other. To get this going, open your app/views.py file and paste the code below to pass the chatbox name from the URL to the HTML file (chatbox.html):

from django.shortcuts import render

def chat_box(request, chat_box_name):
    # we will get the chatbox name from the url
    return render(request, "chatbox.html", {"chat_box_name": chat_box_name})

Now, replace the code you have in project/urls.py with the following code. This will handle the chatbox name stated in the browser (http://127.0.0.1:8002/chat/**chatboxname**/).

from django.contrib import admin
from django.urls import path
from app.views import chat_box

urlpatterns = [
    path("admin/", admin.site.urls),
    path("chat/<str:chat_box_name>/", chat_box, name="chat"),
]

Next, let’s start working on the consumers. Consumers in channels help structure your code as a series of functions to be called whenever an event happens. Consumers are usually written in asynchronous Python. To start, create a new file in app/ folder named consumers.py and paste the code shown below. What the following code is doing is handling what happens when the server connects, disconnects, receives a request, or sends a text.

import json
from channels.generic.websocket import AsyncWebsocketConsumer

class ChatRoomConsumer(AsyncWebsocketConsumer):
    async def connect(self):
        self.chat_box_name = self.scope["url_route"]["kwargs"]["chat_box_name"]
        self.group_name = "chat_%s" % self.chat_box_name

        await self.channel_layer.group_add(self.group_name, self.channel_name)

        await self.accept()

    async def disconnect(self, close_code):
        await self.channel_layer.group_discard(self.group_name, self.channel_name)
    # This function receive messages from WebSocket.
    async def receive(self, text_data):
        text_data_json = json.loads(text_data)
        message = text_data_json["message"]
        username = text_data_json["username"]

        await self.channel_layer.group_send(
            self.group_name,
            {
                "type": "chatbox_message",
                "message": message,
                "username": username,
            },
        )
    # Receive message from chat room group.
    async def chatbox_message(self, event):
        message = event["message"]
        username = event["username"]
        #send message and username of sender to websocket
        await self.send(
            text_data=json.dumps(
                {
                    "message": message,
                    "username": username,
                }
            )
        )

    pass

This Python code defines a Django Channels consumer that manages Django WebSockets connections for a real-time chat application. The ChatRoomConsumer class inherits from AsyncWebsocketConsumer and handles the complete lifecycle of a Django WebSocket connection. When a user connects, the connect method extracts the chatbox name from the URL parameters, creates a unique group name for that chat room, adds the user's channel to that group (allowing multiple users to join the same chat room), and accepts the WebSocket connection. When a user disconnects, the disconnect method cleanly removes their channel from the group to prevent memory leaks and ensure they no longer receive messages.

The message handling happens through two coordinated methods that enable broadcasting to all users in a chat room. The receive method is triggered whenever a message arrives from a client's WebSocket. It parses the JSON data to extract the message text and username, then uses the channel layer to broadcast this data to everyone in the group. The chatbox_message method receives these broadcast messages and forwards them back to the individual WebSocket connections, ensuring that when one user sends a message, all other users in the same chat room receive it instantly. This pattern creates a publish-subscribe system where messages are distributed to all active participants in real-time. Now, let’s add the code for routing.py, which was mentioned earlier. Create a file in your /project folder named routing.py and paste the following code:

from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from django.urls import re_path
from app import consumers

# URLs that handle the WebSocket connection are placed here.
websocket_urlpatterns=[
                    re_path(
                        r"ws/chat/(?P<chat_box_name>\w+)/$", consumers.ChatRoomConsumer.as_asgi()
                    ),
                ]

application = ProtocolTypeRouter( 
    {
        "websocket": AuthMiddlewareStack(
            URLRouter(
               websocket_urlpatterns
            )
        ),
    }
)

Next, let’s build the frontend for the application. Create a file in app/templates with the name chatbox.html, and then paste the code shown below. Most of this code is the starter template code from b Bootstrap, just to give the application some styling.

<html lang="en">

<head>
    <!-- Required meta tags -->
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

    <!-- Bootstrap CSS -->
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css"
        integrity="sha384-JcKb8q3iqJ61gNV9KGb8thSsNjpSL0n8PARn9HuZOnIxN0hoP+VmmDGMN5t9UJ0Z" crossorigin="anonymous">

</head>

<body>

    <div class="container">
        <div class="row d-flex justify-content-center">
            <div class="col-3">
                <form>
                    <div class="form-group">
                        <label for="exampleFormControlTextarea1" class="h4 pt-5">Chatbox</label>
                        <textarea class="form-control" id="chat-text" readonly rows="10"></textarea><br>
                    </div>
                    <div class="form-group">
                        <input class="form-control" placeholder="Enter text here" id="input" type="text"></br>
                    </div>
                    <input class="btn btn-primary btn-lg btn-block" id="submit" type="button" value="Send">
                </form>
            </div>
        </div>
    </div>
    {% comment %} Get data for username and chatbox name{% endcomment %}
    {{ request.user.username|json_script:"user_username" }}
    {{ chat_box_name|json_script:"room-name" }}
    

    <!-- Optional JavaScript -->
    <!-- jQuery first, then Popper.js, then Bootstrap JS -->
    <script src="https://code.jquery.com/jquery-3.5.1.slim.min.js"
        integrity="sha384-DfXdz2htPH0lsSSs5nCTpuj/zy4C+OGpamoFVy38MVBnE+IbbVYUew+OrCXaRkfj" crossorigin="anonymous">
    </script>
    <script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.1/dist/umd/popper.min.js"
        integrity="sha384-9/reFTGAW83EW2RDu2S0VKaIzap3H66lZH81PoYlFhbGU+6BZp6G7niu735Sk7lN" crossorigin="anonymous">
    </script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"
        integrity="sha384-B4gt1jrGC7Jh4AgTPSdUtOBvfO8shuf57BaghqFfPlYxofvL8/KUEfYiJOMMV+rV" crossorigin="anonymous">
    </script>
</body>

</html>

The HTML above creates a simple chat interface using Bootstrap for styling. The page features a centered chatbox with a textarea that displays the chat log messages in read-only mode, an input field where users can type their messages, and a send button to submit them. The layout is responsive and uses Bootstrap's grid system to center a 3-column-wide form on the page. The design is clean and minimal, focusing on functionality with standard Bootstrap components like form controls and a primary button.

At the bottom of the HTML, there are two Django template tags that inject important data into the page as JSON scripts. These tags capture the current user's username and the chatbox/chat room name, making them accessible to JavaScript code that will be added later. The page also includes all necessary Bootstrap dependencies (jQuery, Popper.js, and Bootstrap JS) loaded from CDNs. The next step involves adding custom JavaScript code above these script tags to handle WebSocket connections and real-time message functionality, enabling the WebSocket chat application to send and receive messages dynamically.

Next, we’ll develop the JavaScript code that will fetch the data and handle the WebSocket connection from the frontend side. Paste the following code above the first <script> tag in the HTML file that you just created.

<script>
   const user_username = JSON.parse(document.getElementById('user_username').textContent);
   document.querySelector('#submit').onclick = function (e) {
      const messageInputDom = document.querySelector('#input');
      const message = messageInputDom.value;
      chatSocket.send(JSON.stringify({
          'message': message,
          'username': user_username,
      }));
      messageInputDom.value = '';
   };

   const boxName = JSON.parse(document.getElementById('box-name').textContent);
   # Create a WebSocket in JavaScript.
   const chatSocket = new WebSocket(
      'ws://' +
      window.location.host +
      '/ws/chat/' +
      boxName +
      '/'
   );

   chatSocket.onmessage = function (e) {
      const data = JSON.parse(e.data);
      document.querySelector('#chat-text').value += (data.message + ' sent by ' + data.username   + '\n') // add message to text box
   }
</script>

This JavaScript code handles the real-time chat functionality by managing WebSocket communication between the client and server. First, it retrieves the current user's username from the JSON script tag embedded in the HTML and sets up a click event listener on the send button. When the user clicks send, it captures the message from the input field, packages it along with the username into a JSON object, sends it through the WebSocket connection, and then clears the input field for the next message.

The second part establishes the WebSocket connection itself by extracting the chatbox name from the page and constructing a WebSocket URL using the current host and a specific chat endpoint pattern. It then sets up a message handler that listens for incoming messages from the server. Whenever a new message arrives through the WebSocket, it parses the JSON data and appends the message along with the sender's username to the chat textarea, creating a simple but functional real-time chat experience where users can see messages as they're sent. Now, run the following commands to migrate the authentication model so that you can create new users to test the application:

python manage.py migrate

You can create new users by running the following:

python manage.py createsuperuser

Finally, you can test the chat app by running it and logging in to two users using Django admin. You will be able to do this by logging in to each of the users on different browsers. Then, open the URL 127.0.0.1:8000/chat/newbox/ on each of the browsers, and when you send a text, each user receives the text in real time.

Building on the knowledge gained

In this article, you've learned the fundamentals of WebSocket technology and why it’s essential for building real-time applications like a modern chat app. You saw how Django Channels bridges Django’s traditional HTTP request-response model with persistent WebSocket connections, using the Asynchronous Server Gateway Interface (ASGI) to support real-time communication. By building a functional chat room, you gained hands-on experience with consumers, routing via ProtocolTypeRouter, and managing WebSocket connections from the browser using JavaScript.

Although we successfully built a real-time chat application, this is just the foundation. One of the most important next steps is adding a database connection to persist chat messages and chat logs, allowing users to retrieve conversation history when they rejoin a chat room. This turns a simple demo into a production-ready chat system. For more performance, replacing the in-memory channel layer with Redis as the message broker is strongly recommended. You could also look into how to handle exception in the chat app.

You can further enhance the experience by implementing features such as typing indicators, read receipts, online presence tracking, timestamps for chat messages, and file sharing. At this stage, middleware like AuthMiddlewareStack becomes important for securing WebSocket connections and ensuring authenticated users can only access authorized chat rooms.

With these foundations in place, you’re well-equipped to build advanced real-time systems and Django chat apps using Django Channels.

The best observability platforms for developers

2026-03-10T00:00:00+00:00

At some point, logs stop being enough. As applications grow more distributed, understanding what's actually happening in production becomes harder. That's what observability platforms are built for. The hard part is figuring out which one is actually right for your application — and your budget.

This guide covers some popular options: what they do well, where they fall short, and who they're for. Whether you're evaluating an observability tool for the first time or reconsidering your current tools, the goal is to give you enough signal to make the right choice.

What is an observability platform?

Observability tools and platforms are centered around helping engineers understand system performance, visualize data, and ultimately provide comprehensive visibility across the entirety of an organization's systems.

Software applications used to be built with monolithic code bases. This means that the entire application shared one code base, one deployable artifact, and one runtime. Everything was tightly coupled together. While yes, there was a need for typical metrics and logs to analyze data and system performance, there weren't multiple complex systems to aggregate said data sources.

As the internet continued to take over the world, and more and more complex software companies started to take form, a new paradigm of software architecture emerged. In the early 2010s, the concept of "microservices" started to gather steam as the disadvantages of monoliths, such as slower builds, deployment coupling, and harder to maintain modularity, started to create too much friction across engineering teams.

In comparison to a monolithic code base, microservices look to break apart business logic into small, independent services that each can be deployed separately. For example, if you were looking to build an e-commerce apparel company, you may want to have separate microservices for orders, inventory, and customers. The advantages of microservices are numerous, especially for apps that need to scale, and include independent scaling (you can put more resources behind ONE service without having to scale the entire system) and deploys, fault isolation (an issue in one microservice may not bring down the entire application), and stack freedom (one app can be written in Swift, one in Rails, etc.).

However, the existing tools and solutions for observability no longer met the needs of companies utilizing a microservices architecture. With truly distributed systems running in cloud environments, a need for tools that could provide comprehensive insights and visibility started to arise. Software engineering teams needed one unified platform to observe system behavior across all of their infrastructure components.

For example, let's go back to the e-commerce apparel company. The engineers need to be able to have unified observability to be able to identify overall system performance bottlenecks across the entire stack. Let's say the business metric of the number of orders sees a dramatic drop, and the business wants to understand why. A data observability platform might show that an increase in calls to the authentication service is causing increased latency for customers trying to sign in to their accounts, which is causing clients to get frustrated and abandon orders. How does an observability tool capture this telemetry data?

In software engineering terms, you may hear the term "telemetry data" — this typically includes:

Metrics (numbers over time)
Logs (text events)
Traces (end-to-end request flows)
Events (discrete actions like deploys, errors, or configuration changes)

By bringing together metrics, logs, and traces as observability data, organizations are able to go beyond traditional monitoring to optimize performance, receive real-time, actionable insights, and ultimately reduce operational costs and provide better user interactions.

Types of observability platforms

Not all observability platforms are solving the same problem. Some platforms are built for complex distributed systems: broad feature coverage, enterprise scale, and pricing to match. Developer-focused tools prioritize fast setup and clear, actionable output, with tighter scope and more predictable costs. Logging platforms are built around ingesting and querying massive volumes of log data. Most teams land somewhere on the spectrum between "we need everything" and "we need something that just works."

The features these platforms offer can vary wildly, and the enterprise-focused tools in particular can feel overwhelming. You probably care about a subset of the common features they provide:

APM (Application Performance Monitoring): Provides deep visibility into application code across the stack, including features such as distributed tracing, flame graphs, endpoint latency breakdowns, service dependency maps, and profiling of things like CPU, memory, etc.
Infrastructure and Database Monitoring: These observability tools allow engineers to keep tabs on their CPU, memory, containers, cluster health, and network and host-level metrics. Engineers can also glean real-time insights into the database performance.
Log Management: Observability platforms centralize logs across all sources (application, job processor such as Sidekiq, databases such as Postgres, in-memory store such as Redis, and other cloud services). Engineers are able to see live tails (real-time log streaming), log pipelines (filter, transform, mask), and analyze log data to gain comprehensive insights.
Distributed Tracing: One of the key components that make observability tools able to provide deeper insights, observability tools allow engineers to easily observe the end-to-end journey of requests and identify bottlenecks.
Real User Monitoring (RUM): RUM tooling allows engineers to monitor actual users on their application. For example, an engineer would be able to look up a specific client and view their individual page load times, any errors, and even replay their actual session.
Synthetic Monitoring: Whereas RUM tools are tracking real users' behavior, "synthetic" monitoring is exactly what you would expect it to be — observability tools simulate user behavior using automated checks like typical API calls and frequent user flows like sign-in, login, etc. This can be very beneficial to continuously monitor the state of the application.
Dashboards: Oftentimes, it is helpful for engineers to pull insights from multiple tools. For example, an engineer might want to look at the performance of a specific endpoint, alongside business metrics, alongside SLO metrics. Observability tools often provide customizable dashboards so that engineers can gain this comprehensive visibility by creating graphs, time comparisons, and views that can include logs, metrics, and traces.
Alerting & SLO Monitoring: These tools help engineers ensure they are being notified of potential issues. For example, observability tools may allow a team to create an alert that pages someone if a Sidekiq queue is greater than a certain amount over a certain time period.
Security Monitoring: Observability tools often provide security analytics to identify potential threats or vulnerabilities.
CI (Continuous Integration) Visibility: CI is a new-ish software development practice in which teams merge their code changes into a shared main branch, and each merge automatically triggers the build, the test suite to run, any linting or code quality checks, etc. Most observability tools know this is common practice for most dev teams, and provide monitors so that engineers can keep track of pipeline duration, build failures, and metrics related to their test suite — for example, how quickly their tests run and any flaky tests (tests that fail or pass sporadically, and are unrelated to the actual code changes being made).
Integration Ecosystem: In order to make an observability platform the right observability tool for your organization, you need to ensure that it can integrate with all of your existing stack. For example, if you use AWS, you will want to make sure you choose an observability tool that can integrate with AWS. This is, of course, a bit of a trivial example given the popularity of AWS.

Six great observability tools

Now that we have a grasp of what observability tools are and why they are important for system performance, especially in distributed systems environments, the next step in our journey is to look at who the players are in this space.

1. Datadog

Overview: Arguably the most popular tooling with an incredible overall market share, Datadog is likely a name in this industry that you are already familiar with. Datadog's centralized platform provides logs, metrics, traces, APM (application performance monitoring), and infrastructure monitoring. It is often the top choice for cloud environments that utilize a heavy microservices architecture.

Key Features:

APM (Application Performance Monitoring)
Infrastructure and database monitoring
Log management
Distributed tracing
Real user monitoring (RUM)
Synthetic monitoring
Dashboards
Alerting & SLO monitoring

Pricing: Datadog's pricing model is based on usage and depends on a few different levers, such as the number of hosts, data volume, and features that you choose to integrate. For basic infrastructure monitoring, you are charged per "host" — typically around $15 per host per month. If you enable additional features, such as APM, your cost goes up to around $30 per host per month. For other services, pricing is based on how much you use them. For example, per metric, per GB of ingested log data, etc. One of the downsides of Datadog is that its pricing model can be quite complex.

Overall Pros: Trusted by many large companies, strong real-time debugging and dashboarding/visualization tools, full product suite.

Cons: Can be expensive, and the pricing model itself can be complex, may be overkill for smaller companies.

2. New Relic

Overview: Another strong player in the observability tools space, New Relic, is also highly sought after by larger companies. It also includes a full product suite that ingests telemetry data (logs, metrics, traces, events) and allows engineers to monitor system performance and visualize data.

Key Features:

APM (Application Performance Monitoring)
Infrastructure and database monitoring
Log management
Distributed tracing
Real user monitoring (RUM)
Synthetic monitoring
Dashboards

Pricing: New Relic's pricing is usage-based, centered primarily on data ingestion and user "seats" (i.e, how many users will be utilizing their tools). New Relic does have a free tier, but after that, additional usage is based on GB of data ingested. "Seats" include "full platform users" who need access to advanced observability features, and "basic users" who are free.

Overall Pros: Strong integration support (780+), offers "intelligent observability" which utilizes AI to predict issues and automate workflows, and offers a "free tier."

Cons: Utilizes its own query language (NRQL), which creates a steep initial learning curve, and does not have as strong infrastructure tooling as other options.

3. Dynatrace

Overview: Dynatrace is an enterprise-grade observability and automation platform known for its AI-driven insights. It is often the best observability tool for organizations operating large distributed systems in multi-cloud or Kubernetes-heavy environments.

Key Features:

APM (Application Performance Monitoring)
Infrastructure and database monitoring
Log management
Distributed tracing
Real user monitoring (RUM)
Synthetic monitoring
Dashboards

Pricing: Dynatrace's pricing model centers around host units and data volume, dependent on which modules are being used. It is one of the more expensive options in the industry.

Overall Pros: Less manual setup, known for being "best in class" for AI-assisted root cause analysis.

Cons: Very expensive, more complex procurement and onboarding.

4. Honeybadger

Overview: Honeybadger is a developer-focused observability platform built around the things most teams actually need: error tracking, uptime monitoring, cron & heartbeat monitoring, application performance, and log management. Setup takes minutes. You can send structured event data and query it directly, with automatic dashboards and alerting on top. It's designed to give developers direct access to their data without black-box abstractions or a dedicated ops team to run it.

Key Features:

Error and exception tracking
Uptime and cron/heartbeat monitoring
Performance insights
Log management and event tracking
Querying, dashboards, and alerting

Pricing: Honeybadger's pricing is straightforward based on data volume, with no cap on users or hosts. It is typically much more affordable than other observability tools that tier on those dimensions.

Overall Pros: Fast, easy setup; strong documentation and integrations with common tools like Slack and GitHub, and simple and transparent pricing.

Cons: Not the right fit if you're running a heavily distributed microservices architecture. Deep infrastructure monitoring requires more manual setup compared to dedicated enterprise platforms.

5. Splunk

Overview: Splunk is another popular option that was originally most known for its log indexing and search, but expanded into a full observability suite that includes all of the typical features. It is known for its deep log management capabilities and ability to handle massive volumes of machine-generated log data.

Key Features:

Log Management and Search
APM (Application Performance Monitoring)
Infrastructure and Database Monitoring
Distributed Tracing
Real User Monitoring (RUM)
Synthetic Monitoring
Dashboards

Pricing: The pricing varies depending on whether or not your team utilizes their traditional log product or their full observability platform. For their observability tool, pricing is typically based on the number of hosts.

Overall Pros: Supports open standards such as OpenTelemetry, highly customizable dashboards, and ties into Splunk's other offerings if you already utilize them.

Cons: Higher learning curve given its feature set, still can be cost-prohibitive depending on your host count, often overkill for smaller projects and teams.

6. Sentry

Overview: Sentry is a developer-focused observability platform that is popular for its error and performance monitoring. It has a strong following with product engineering teams who want to better understand how code changes impact application health and user experience.

Key Features:

Error Tracking & Alerting
APM (Application Performance Monitoring)
Release & Deployment Health Analytics
Distributed Tracing
Dashboards
Session & User Impact Visibility

Pricing: Pricing is based on a combination of event-based usage and plan tiers. There is a free tier available. Similar to other platforms, pricing increases as the volume of events grows.

Overall Pros: Sentry has a generous free tier that can be utilized for smaller companies or if you just want to try it out. Sentry also offers a strong integration ecosystem and places a heavy emphasis on helping developers accelerate root cause analysis and triaging issues.

Cons: Does not provide a full observability suite like Datadog and New Relic do, so it may not be relevant for larger companies. Their distributed tracing is also not as robust as some of the other players outlined in this article.

What to look for in an observability tool

So, you may be asking yourself — when do I need to start thinking about a data observability tool, and how do I choose? Typically, you may start thinking about choosing an observability tool when you are no longer able to utilize basic logs and ad-hoc debugging to identify and solve problems in production.

Maybe you have started to utilize background jobs or have introduced asynchronous workflows. Maybe you have started down the path of breaking down a monolithic code base into microservices.

The main red flag indicating that you may need to add an observability tool to your stack is if you (and your team) start noticing an increasing amount of time that it takes to resolve incidents, and feel like you do not have a true understanding of how your code behaves in production.

OK, so you have decided that you need a unified observability platform now that traditional monitoring tools are not up to snuff. How do you choose the right fit? Here are some questions to consider:

What type of environment does my application run in?

For example, if you are running distributed systems with microservices, you will want to prioritize platforms with strong distributed tracing and service dependency mapping.

What are my biggest pain points?

When considering the most effective observability tool for your team, think about where your problems lie. Is your biggest pain point debugging failures or log management? Then, you may want to focus on a data observability tool with advanced log indexing and search.

How big is my team?

What observability tool is right for you will also heavily depend on the size of your team. If your team is small, you may want to prioritize tools that are more lightweight and easy to set up. If, however, you are part of a multi-service enterprise organization, you will likely need an observability tool that offers tools that can be used in complex environments and have a strong reputation in the industry — the last thing you want to worry about is your observability tool going offline or having bugs of its own.

What is my budget?

And then, there's money. Ultimately, while you may love to have the shiniest observability tool with the most advanced feature set, it may not realistically be in your budget. You also need to consider pricing structures... if your application spits out a large amount of log data from lots of different data sources, platforms whose pricing is based on data ingestion may become extremely expensive very quickly.

Reflecting on the questions above should help point you in the right direction.

When is Honeybadger observability the right fit?

We built Honeybadger for teams who are tired of paying for features they don't use, and want a simpler alternative. If you run a monolith or a reasonable number of distributed services and want visibility into errors, downtime, and application performance — without a complex setup or a surprising bill — Honeybadger might be the right fit.

It's not the right tool if your primary need is deep infrastructure monitoring across a large microservices architecture. For that, Datadog or Dynatrace will serve you better. But if you want something that works out of the box, surfaces actionable issues, and doesn't require a dedicated team to maintain, give Honeybadger a try.