Monitoring Sidekiq | An Actionable Guide

Background processing is a common task in many applications. Anytime you want to kick off a long-running job without blocking your application’s main thread, a background processing tool should be the first thing you reach for—there’s no need to build a complete distributed system before you’re sure you need it.

Sidekiq is one of the most popular background processing tools written in Ruby. Its simple API and performance make it an excellent choice for developers looking for an easy way to add background job processing to their app.

In this article, you’ll learn more about the importance of monitoring your Sidekiq jobs, and explore a tutorial on how to do so using Cronitor.

Sidekiq Basics

Sidekiq uses threads to process background jobs. This allows it to take advantage of multiple cores on a machine and makes it faster, in many cases, than other Ruby background processing tools that use operating system processes instead of threads. Retries are built in. Rate limiting is a Sidekiq Enterprise feature.

Some of the top use cases for Sidekiq include the following:

• Sending emails in the background
• Generating reports
• Processing images
• Importing and exporting data

Overall, Sidekiq is an excellent choice for developers looking for an easy-to-use and fast background processing tool written in Ruby.

However, if you run Sidekiq, it’s a good idea to consider how you’ll monitor it. After all, you wouldn’t want a background job to fail because your Sidekiq queue has stopped working properly without you realizing it.

Sidekiq Monitoring

Monitoring Sidekiq is important because you want to ensure that your app’s background jobs run and complete as expected. Some of the reasons a job might fail include the following:

• It never started in the first place. In this case, you’ll notice that your Sidekiq jobs queue is not growing when you expect it to.
• It crashed partway through processing. If this happens, you will notice that Sidekiq jobs are starting but not completing.
• It got stuck in an infinite loop and ate up all of your server’s resources.

You can monitor several related metrics to spot problems with Sidekiq before they become critical, like the following:

• Queue size: The number of jobs in the queue is a good indicator of Sidekiq’s health. If the number of jobs is larger than normal, it could indicate that Sidekiq is not processing them fast enough. It could also indicate that your app is malfunctioning or experiencing a DDoS attack. On the other hand, if the number of jobs in the queue drops to zero, it could be an indication that Sidekiq has stopped processing jobs altogether. This may be due to a problem with Sidekiq, or it might mean that your application is down.
• Queue backups: If the number of jobs in the queue starts to grow rapidly, it could mean that Sidekiq is not able to keep up with the rate at which new jobs are being added. This could be due to a sudden increase in the number of jobs being added. While this isn’t necessarily a problem if Sidekiq has a chance to catch up before the queue grows too large, it’s worth watching as it may be a sign you need to scale Sidekiq up to keep up with your application’s demands.
• Queue-size anomalies: If the number of jobs in the queue suddenly spikes or drops sharply, it could also be an indication of a problem. For example, if the queue size suddenly spikes, it could mean that Sidekiq has stopped processing jobs. If the queue size suddenly drops, it could mean that Sidekiq is no longer receiving new jobs. The key thing to look for is whether the queue size is outside the range you’d expect to see when your application is functioning normally.
• Queue latency: The time it takes for a job to go from being added to the queue to being processed by Sidekiq is known as the queue latency. The queue latency Sidekiq reports is the age in seconds of the oldest job currently in the queue. If the queue latency starts to increase, it could be an indication that Sidekiq is having difficulty processing jobs in a timely manner.

Aside from monitoring your Sidekiq queue, you should also monitor the background tasks themselves. Specifically, you should monitor two things in particular: job performance and job frequency.

• Job performance: You’ll want to monitor the performance of your background jobs to ensure that they are running as expected and completing in a timely manner. If a job starts to take longer than usual to complete, it could be an indication of a problem. For example, if a job that typically takes one minute to complete starts taking five minutes, it could be an indication that Sidekiq is having difficulty processing it.

• Job frequency: You’ll also want to monitor the frequency of your background jobs. If you notice that a job is being run more frequently than usual, it could be an indication of a problem. For example, if a job that is typically run once a day starts running every few minutes, it could mean that Sidekiq is stuck and is processing the same job over and over again—or it could mean that your application is enqueueing jobs too frequently.

You should also understand the difference between monitoring periodic Sidekiq jobs versus monitoring “regular” Sidekiq background jobs.

• Periodic Sidekiq jobs are background jobs scheduled to run at regular intervals. For example, you might have a periodic Sidekiq job that runs once a day to send a daily digest email. Monitoring the performance of these jobs is important because they usually perform business-critical tasks. Consequently, you want to ensure that they are running as expected and completing within a reasonable time frame.

• Regular Sidekiq background jobs are background jobs not scheduled to run at regular intervals. For example, you might have a regular Sidekiq background job that is triggered when a user signs up for your app. Monitoring the performance of these jobs is important because they often perform tasks that must be completed quickly. For instance, you might use a Sidekiq job to send your users password-reset emails. Users trying to sign in to your app will want to receive these emails quickly, so Sidekiq monitoring, in this case, could alert you to performance issues affecting your app’s user experience.

Sidekiq also comes with a web dashboard that provides an overview of the Sidekiq process. It includes information on the number of jobs in the queue, the queue latency, and various statistics on job performance. The Sidekiq dashboard also provides information on the hostname, pid, and uptime of the Sidekiq process. This information can be useful for debugging purposes and to keep an eye on how healthy your queues are:

Monitoring Sidekiq with Cronitor

Now that you’ve learned why monitoring Sidekiq queues is important, it’s time to learn how to do it. This tutorial will utilize a Rails app since that’s where Sidekiq is most commonly used, but the instructions below will work anywhere you use Sidekiq.

If you’d like to follow along and try adding to the app yourself, you can find it in this GitHub repository. Note that the main branch contains the final version of the app with Cronitor monitoring added, while the pre-Cronitor version is in the pre_cronitor branch.

Prerequisites

You will need the following prerequisites to run the app:

• Ruby and Redis that match the Sidekiq you install. Current Sidekiq requires Ruby 3.2 or newer, and Redis 7.0 or newer (Valkey 7.2 or newer also works). The sample repo's Gemfile still pins Ruby 3.1.2 and Rails 7.0, which is older than that. If bundle install pulls Sidekiq 8, use a Ruby it accepts, or pin Sidekiq to a release that supports the Ruby in the Gemfile.
• Redis on the machine: If you are running Linux or macOS, you can download Redis here. With Docker, docker pull redis:latest and docker run -p 6379:6379 redis:latest is enough.

The Demo App

You can clone the code by running the following:

git clone https://github.com/rpeden/cronitor-sidekiq

After Git finishes cloning the code, run the following to switch to the right branch and get the application ready to run:

cd cronitor-sidekiq
git checkout pre_cronitor
bundle install
rails s

In a separate terminal window, switch to the directory you cloned the app into and run the following:

bundle exec sidekiq -C config/sidekiq.yml

The Rails development server will start in one terminal window, and Sidekiq will start in the other.

Open a web browser and navigate to http://localhost:3000, and you’ll see a simple app that simulates a web page downloader. That’s exactly the kind of time-consuming task you don’t want blocking a web app thread, making it the perfect candidate for Sidekiq. When you’re done, press Ctrl+C in both windows to stop the applications.

Since a full Sidekiq tutorial is beyond the scope of this article, Sidekiq itself is already in place. In particular, note this line added to config/application.rb:

config.active_job.queue_adapter = :sidekiq

This tells Rails to use Sidekiq to run any ActiveJob jobs the app creates.

The app has also set some Sidekiq queue configuration parameters in config/sidekiq.yml:

development:
  :concurrency: 5

production:
  :concurrency: 10

:max_retries: 1

:queues:
  - default

And it does some basic configuration of Sidekiq in config/initializers/sidekiq.rb:

Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1') }
end
Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1') }
end

Although this isn’t a full tutorial, it’s essentially everything you need to do to add Sidekiq to a Rails app.

Finally, open app/jobs/save_page_job.rb for the simulated page job that Sidekiq must handle:

class SavePageJob < ApplicationJob
  queue_as :default
  sidekiq_options cronitor_key: "save-page-job"

  def perform(*args)
    # Simulates a long page crawl and save
    p "Starting simulated page download!"
    sleep 10
    p "If this were a production app, we would have saved the page at #{Time.now().strftime('%F - %H:%M:%S.%L')}"
  end
end

Adding Cronitor job monitoring

Sign up if you do not already have an account. In the app, open Get Started, then Jobs, then Sidekiq. The panel is the Sidekiq Integration Quickstart.

The quickstart is four steps. A shorter Ruby overview is in Scheduling Cron Jobs in Ruby. The gem readme is sidekiq-cronitor.

1. Install the gem. gem install sidekiq-cronitor, or add gem 'sidekiq-cronitor' to the Gemfile and run bundle install.

2. Set the API key. In production, export CRONITOR_API_KEY before Sidekiq starts. The quickstart also shows Cronitor.api_key = in Ruby. Both work. Use an SDK Integration key from Settings, then API. Do not commit the key.

3. Register the server middleware, usually in config/initializers/sidekiq.rb. Cronitor then reports each job as it runs and creates a monitor on the first event if one does not exist.

   require 'sidekiq/cronitor'
   
   Sidekiq.configure_server do |config|
     config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1') }
     config.server_middleware do |chain|
       chain.add Sidekiq::Cronitor::ServerMiddleware
     end
   end
   
   Sidekiq.configure_client do |config|
     config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1') }
   end
   

   The monitor name is cronitor_key on the payload when that is set, otherwise a key in a cronitor hash, otherwise class. A native Sidekiq job uses its class name. ActiveJob runs inside a wrapper, so without cronitor_key those jobs share the wrapper's monitor. sidekiq_options cronitor_key: "save-page-job" on SavePageJob is copied into the payload, and the Jobs card uses that name.

4. Optional, and only if you schedule jobs. Sidekiq Enterprise periodic jobs:

   Sidekiq::Cronitor::PeriodicJobs.sync_schedule!
   

   The sidekiq-scheduler gem:

   Sidekiq::Cronitor::SidekiqScheduler.sync_schedule!
   

   Those calls copy the schedule onto the monitor. Skip them if every job is enqueued by the app rather than by a clock.

Restart Rails and Sidekiq the same way as before (rails s, and bundle exec sidekiq -C config/sidekiq.yml). Open http://localhost:3000 and submit a few URLs so SavePageJob runs.

Then open Jobs. Each card shows Duration, Expected, Last Event, and Last Issue. For this app the card is save-page-job.

Open the job for Status, Success, Performance, Executions, Alerts, and Latest Activity.

A job that is not periodic has no schedule until you set one, or until sync_schedule! writes it. Edit is where Notify, grace period, and failure tolerance live. If you do not pick a notification list, Cronitor uses the account default. The list in this shot is named Standard Alert.

Adding queue monitoring

Job telemetry does not tell you that the queue itself is stuck. Sidekiq Web already has a JSON stats endpoint. Mount it in config/routes.rb:

require 'sidekiq/web'

Rails.application.routes.draw do
  root "page_saver#index"
  post "/save_page", :to => "page_saver#create"
  mount Sidekiq::Web => "/sidekiq"
end

http://localhost:3000/sidekiq is the dashboard from earlier. http://localhost:3000/sidekiq/stats is JSON. It includes sidekiq.enqueued, sidekiq.default_latency, and store_name and store_version on the redis object:

{
  "sidekiq": {
    "processed": 13,
    "failed": 0,
    "busy": 0,
    "processes": 1,
    "enqueued": 0,
    "scheduled": 0,
    "retries": 0,
    "dead": 0,
    "default_latency": 0
  },
  "redis": {
    "redis_version": "7.0.2",
    "uptime_in_days": "0",
    "connected_clients": "1",
    "used_memory_human": "1.48M",
    "used_memory_peak_human": "1.56M",
    "store_name": "Redis",
    "store_version": "7.0.2"
  },
  "server_utc_time": "03:03:31 UTC"
}

Secure Sidekiq Web before it is reachable beyond your laptop. The sample app skips that so the local steps stay short.

Cronitor's Checks run from the public internet, so they cannot see localhost. Two ways to close that gap:

• On a machine that can already reach Sidekiq Web, follow monitor private endpoints. A cron job inside the network requests /sidekiq/stats and Cronitor watches that job. The Sidekiq port stays closed to the internet. This is the production approach.
• On a laptop, for a short test, a tunnel such as ngrok publishes the Rails port. The tunnel dies when you stop it, and it exposes Sidekiq Web, so do not leave it up. The rest of this section is that local test, not the setup to copy into production.

After ngrok's own install, run:

ngrok http 3000

ngrok prints a forwarding URL:

Rails rejects a host it does not recognize. Allow the host ngrok printed. Free tunnels commonly end in ngrok-free.app. Older ones used ngrok.io. In config/environments/development.rb:

config.hosts << /.*\.ngrok-free\.app/
config.hosts << /.*\.ngrok\.io/

Restart rails s.

Open Checks and choose Create Check. Name it, then under Request Details set the endpoint to the ngrok URL plus /sidekiq/stats.

Assertions are the healthy condition, on the same form. response.json with key sidekiq.enqueued compares that field. A backlog alert looks like sidekiq.enqueued < 100: the check fails when the queue grows past that. sidekiq.enqueued >= 1 does the opposite of a backlog alert. It fails while the queue is empty and passes once a job is waiting, so it is a poor test of "Sidekiq is unhealthy." To force a failure on purpose, pick a condition the current JSON does not meet, save, and wait for the next check.

Alert Settings is the Notify list, an optional note, reminders, and failure tolerance. Save Monitor.

After a failed assertion, the check card on Checks leaves the healthy state, and the check page lists the failed request under Latest Activity.

Conclusion

Sidekiq Web shows the queue. It does not tell you that a job class stopped running. The Sidekiq Integration Quickstart (the gem, CRONITOR_API_KEY, and Sidekiq::Cronitor::ServerMiddleware) is the job path. For the stats URL, prefer a check that runs inside the private network, and treat ngrok as a local experiment. Sign up if you want to try it on your own apps.