ActiveStorage: How to add a new preprocessed named version

Given there is a user with an attachable avatar:

class User < ApplicationRecord
  has_one_attached :avatar
end

If you want to add a preprocessed version follow these steps:

  1. Add the named version and deploy
class User < ApplicationRecord
  has_one_attached :avatar do |attachable|
    attachable.variant :preview, resize_to_fit: [177, 177 * 9 / 16], preprocessed: true
  end
end
  1. Preprocess this version for all existing records `bundle exec rails runner 'User.find_each { |user| user.avatar.variant(:preview).proc...

Debugging Capistrano

Capistrano 3 has a doctor task that will print information about

  • Environment: Ruby, Rubygems and Bundler versions
  • List of Capistrano gems and whether an update is available
  • All config variables and their values
  • Capistrano server config
$ bundle exec cap staging doctor

Capistrano task to edit staging / production credentials

When using Rails credentials, you will edit the encrypted credentials for staging or production environments from time to time. To do that you need the secret key which should only live on the servers.

Do not download these key files to your local dev environment. They are sensitive and must not be stored on your machine.

Instead, put the attached capistrano task into lib/capistrano/tasks/ of your application. It expects environment specific keys to live in :shared_path/config/credentials/:stage.key. If you have a single master.key...

Capistrano: Configure environment specific array attributes

Using Capistrano, we usually have some array configurations in the config/deploy.rb file, like set :linked_files, %w[config/database.yml], so in this case we don't have to manage the database configuration individually on every server.

In a specific case, one of our projects supports sign in by SAML, but not every deploy target has this feature activated. Here comes a nice handy Capistrano feature, which lets us modify the default configuration for individual env...

Migrating from Elasticsearch to Opensearch: Overview

Why do we migrate?

Due to a change in licensing, we cannot provide Elasticsearch versions >= 8.0.
Version 7.17.x will reach EOL status with the release of Elasticsearch version 9.
We have decided to use OpenSearch as a replacement, since it is a fork of Elasticsearch version 7.10.2, still running under the previous licensing model and wire-compatible.
A more detailed reasoning can be found on their [website](https://opensearch.o...

Capistrano 3: Running a command on all servers

This Capistrano task runs a command on all servers.

bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'

Code

# lib/capistrano/tasks/app.rake

namespace :app do

  # Use e.g. to grep logs on all servers:
  #   b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log' 
  #
  # * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
  # * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
  # * To be able to process ...

Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)

General

A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch

This card deals with specifics concerning the use of searchkick.

Step 1: Make Opensearch available for Searchkick

In your Gemfile

# Search
gem 'searchkick'                   # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'

in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:

SEARCHKICK_CLIENT_T...

Capistrano: creating a database dump if migrating

In Capistrano 3, your Capfile requires 'capistrano/rails/migrations', which brings two Capistrano tasks: deploy:migrate and deploy:migrating. The former checks whether migrations should be performed. If so, the latter is invoked, which performs the actual migrations.

Knowing this, it is easy to dump the db only if migrations will run. First, enable conditional migrations:

# config/deploy.rb
set :conditionally_migrate, true # Only attempt migration if db/migrate changed

Then hook up the dump task to deploy:migrating:

How to make sure that manual deploy tasks (scheduled in Pivotal Tracker) are executed on deploy (with Capistrano)

We regularly have tasks that need to be performed around a deploy. Be it to notify operations about changed application behavior, be it to run a little oneline script after the deploy. Most database-related stuff can be handled by migrations, but every once in a while, we have tasks that are much easier to be performed manually.

Writing deploy tasks

Here is how we manage the deploy tasks themselves:

  • Deploy tasks are written inside the Pivotal Tracker story description, clearly marked (e.g. with a headline "Deploy task")
  • We disting...

Local deployment after pipeline succeeds

If you have a fully functional CI pipeline but no CD, you might find yourself frequently waiting for CI (with "merge after pipeline succeeds") just to perform the deployment.

The following command waits for the next commit that lands on the current branch (should be main or similar) and proceeds to deploy staging afterwards:

alias await-deployment='watch -g git pull && bundle exec cap staging deploy'

Note

Use at your own risk.
You could be deploying code from someone else that was pushed to the same branch in the meantime.

Rails: Fixing ETags that never match

Every Rails response has a default ETag header. In theory this would enable caching for multiple requests to the same resource. Unfortunately the default ETags produced by Rails are effectively random, meaning they can never match a future request.

Understanding ETags

When your Rails app responds with ETag headers, future requests to the same URL can be answered with an empty response if the underlying content ha...

Heads up: Deployment with newly generated SSH key (using ED25519) might fail

If you use a newer SSH key generated with the ED25519 algorithm instead of RSA (see Create a new SSH key pair), the deployment with Capistrano may fail with the following message:

The deploy has failed with an error: unsupported key type `ssh-ed25519'
net-ssh requires the following gems for ed25519 support:
 * ed25519 (>= 1.2, < 2.0)
 * bcrypt_pbkdf (>= 1.0, < 2.0)
See https://github.com/net-ssh/net-ssh/issues/565 for more information
Gem::LoadError : "ed25519 i...

Issue Checklist Template

This is a checklist I use to work on issues. For this purpose I extracted several cards related to the makandra process and ported them into a check list and refined that over time a little bit.

This task list is divided by the Gate keeping process in the following steps:

1. Starting a new feature
2. Working on the issue
3. Finishing a feature
4. After Review

Here are some ti...

Fix: esbuild assets are missing after capistrano deploy

Issue: You have an app using jsbundling-rails and esbuild. After deploy, the assets built by esbuild are missing in public/assets.

Solution: Add app/builds to your git repo (by adding a app/builds/.keep file).

Something in sprockets is caching paths and refuses to accept files in "unknown" locations.

net-ssh and openssl-3.0.0

You'll need openssl-3 or newer for servers running 22.04

Ruby version 3.1 uses by default the gem openssl-3.0.0. This can cause issues with the gem net-ssh (6.1.0). This is a known bug.

Typically this can cause an error while deploying an application with capistrano:

could not verify server signature (SSHKit::Runner::ExecuteError)

or

Ed25519::VerifyError: signature verification failed!

As temporary workaround add the following line to your Gemfile:

gem 'openssl', ...

Deployment: Merge consecutive commits without cherry-picking

You want to deploy new features but the latest commits are not ready for production? Then use git merge master~n to skip the n-last commits.

Tip

A big advantage of merging vs. cherry-picking is that cherry-picking will create copies of all picked commits. When you eventually do merge the branch after cherry-picking, you will have duplicate commit messages in your history.

Example

It's time for a production deployment!

git log --pretty=format:"%h - %s" --reverse origin/production..origin/master

0e6ab39f - Feature A
6396...

When does Webpacker compile?

Webpack builds can take a long time, so we only want to compile when needed.

This card shows what will cause Webpacker (the Rails/Webpack integration) to compile your assets.

When you run a dev server

While development it is recommended to boot a webpack dev server using bin/webpack-dev-server.

The dev server compiles once when booted. When you access your page on localhost before the initial compilation, the page may load without assets.

The ...

Capistrano: Deployment issue undefined method `[]' for nil:NilClass

In newer passenger versions the output of passenger -v has changed. capistrano-passenger tries to parse the version and now causes the error undefined method '[]' for nil:NilClass. To fix this you only need to upgrade the capistrano-passenger gem.

Therefore run bundle update capistrano-passenger --conservative.

The version change of passenger from 6.0.7 to 6.0.8 has triggered this problem. This is fixed in capistrano-passenger >= 0.2.1.

Delivering Carrierwave attachments to authorized users only

Preparation

To attach files to your records, you will need a new database column representing the filename of the file. To do this, add a new migration (rails g migration <name>) with the following content:

class AddAttachmentToNotes < ActiveRecord::Migration[6.0]
  def change
    add_column :notes, :attachment, :string
  end
end

Don't forget to rename the class and change the column details to fit your purpose. Run it.

1) Deliver attachments through Rails

The first way is to store your Carrierwave attachments not ...

How to fix "Command "webpack" not found"

I just ran into this deployment error after switching from the asset pipeline to webpack:

01:05 deploy:assets:precompile
      01 bundle exec rake assets:precompile
      01 Compiling...
      01 Compilation failed:
      01 yarn run v1.22.5
      01 error Command "webpack" not found.
rake stderr: Nothing written

The problem is not related to the "webpack" dependency. You probably just forgot to add a binstub to run "yarn install":

Add these lines to "bin/ya...

SSHKit 1.9.0 failure for Capistrano deploy

SSHKit 1.9.0 might fail with the following error, when trying to deploy a Rail application. Upgrading the gem to version 1.21.0 fixed the issue.

Traceback (most recent call last):
	17: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
	16: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9.0/lib/sshkit/backends/abstract.rb:29:in `run'
	15: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9....

Whenever requires you to set the application attribute in the Capistrano config

Whenever requires you to set the application attribute in your Capistrano configuration. Otherwise your cronjobs are created multiple times.

Example entry in config/deploy.rb:

set :application, 'some-app' # allows "set :whenever_identifier, ->{ "#{fetch(:application)}_#{fetch(:stage)}" }" to work as expected

Good

Then the crontab -l output will look like this:

# Begin Whenever generated tasks for: som...

Configuring Webpacker deployments with Capistrano

When deploying a Rails application that is using Webpacker and Capistrano, there are a few configuration tweaks that optimize the experience.

Using capistrano-rails

capistrano-rails is a Gem that adds Rails specifics to Capistrano, i.e. support for Bundler, assets, and migrations. While it is designed for Asset Pipeline (Sprockets) assets, it can easily be configured for Webpacker. This brings these features to the Webpacker world:

  • Automatic removal of expired assets
  • Manifest backups

Carrierwave: How to migrate to another folder structure

A flat folder structure can be cool if you have only a few folders but can be painful for huge amounts. We recently had this issue in a project with more than 100.000 attachments, where we used a structure like this /attachments/123456789/file.pdf.

Even the ls command lasted several minutes to show us the content of the attachments folder.

So we decided to use a more hierarchical structure with a limited maximum of folder per layer. Here are a few tips how to migrate your files to their new...