Add Sidekiq to a Docker Compose managed Rails project

Published: November 17, 2019 by Yi Zeng

Categories:
tutorials 26

Sidekiq is a simple, efficient background processing for Ruby that has a lot of advantage over delayed_job or Resque¹.

Here are the few simple steps to integrate Sidekiq into a Dockerized Rails project:

If the project has not been dockerized yet, the previous post can be helpful.

Add Sidekiq to Gemfile

gem 'sidekiq' # Simple, efficient background processing for Ruby

Build project

docker-compose build

Setup Sidekiq

Follow the steps below to setup Sidekiq.

Generate a Sidekiq worker.

docker-compose run web bundle exec rails g sidekiq:worker Hard

If ActiveJob is used, update config/application.rb.
```
config.active_job.queue_adapter = :sidekiq
```
If ActiveJob is not used, delete app/jobs folder.
```
rm -r app/jobs/
```

Set the Sidekiq Redis URL.

By default, Sidekiq tries to connect to Redis at localhost:6379, which needs to be changed the Redis URL in Docker.

Create config/initializers/sidekiq.rb.
```
touch config/initializers/sidekiq.rb
```

Set URL in initializer.

Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch('REDIS_URL_SIDEKIQ', 'redis://localhost:6379/1') }
end

Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch('REDIS_URL_SIDEKIQ', 'redis://localhost:6379/1') }
end

Add Sidekiq and Redis to docker-compose.yml

version: '3'

services:
  db:
    image: 'postgres:10-alpine'
    volumes:
      - 'postgres:/var/lib/postgresql/data'
    ports:
      - '5432:5432'

  redis:
    image: 'redis:5-alpine'
    command: redis-server
    ports:
      - '6379:6379'
    volumes:
      - 'redis:/data'

  sidekiq:
    depends_on:
      - 'db'
      - 'redis'
    build: .
    command: bundle exec sidekiq
    volumes:
      - '.:/project'
      - '/project/tmp' # don't mount tmp directory
    environment:
      - REDIS_URL_SIDEKIQ=redis://redis:6379/1

  web:
    depends_on:
      - 'db'
      - 'redis'
    build: .
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0' -e ${RAILS_ENV}"
    ports:
      - '3000:3000'
    volumes:
      - '.:/project'
    environment:
      - REDIS_URL_SIDEKIQ=redis://redis:6379/1

volumes:
  redis:
  postgres:

Customize REDIS_URL_SIDEKIQ

Either add as an environment variable in docker-compose.yml:

sidekiq:
  depends_on:
    - 'db'
    - 'redis'
  build: .
  command: bundle exec sidekiq
  volumes:
    - '.:/project'
  environment:
    - REDIS_URL_SIDEKIQ=redis://redis:6379/12

Or add to .env file and load it in docker-compose.yml

OTHER_ENV=example
REDIS_URL_SIDEKIQ=redis://redis:6379/12

sidekiq:
  depends_on:
    - 'db'
    - 'redis'
  build: .
  command: bundle exec sidekiq
  volumes:
    - '.:/project'
  env_file:
    - '.env'

Add UI page to routes.rb.

require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'

Start Server

docker-compose up --build

Go to http://localhost:3000/sidekiq to verify.

Troubleshooting

Errno::ENOENT - No such file or directory - bs_fetch:atomic_write_cache_file:rename

Reason: Two processes are trying to load the /app/tmp directory at the same time².
Solution: Add - '/project/tmp' to Sidekiq volumes in docker-compose.yml.

Sidekiq Comparison ↩
See this GitHub Comment. ↩