Maintaining custom application tasks in Rails

Here are some hints on best practices to maintain your tasks in larger projects.

Rake Tasks vs. Scripts

  • The Rails default is using rake tasks for your application tasks. These live in lib/tasks/*.
  • In case you want to avoid rake for your tasks and just use plain ruby scripts, consider lib/scripts/* as folder.

Keeping tasks slim

For readability and testing it's easier to keep your tasks slim. We suggest to use folders inside the tasks or scripts folder.

Example for a task:

The slim task lib/tasks/gitlab.rb:

Rails: Using custom config files with the config_for method

You can use the config.x configuration in combination with config_for to configure global settings for your Rails 4.2+ application.

Example

In your config/application.rb assign the settings from e.g. config/settings.yml as follows:

module FooApplication
  class Application < Rails::Application
    config.x.settings = config_for(:settings)
  end
end

The config/settings.yml might look as follows:

shared: &shared
  email: info@example.com
...

Disable PostgreSQL's Write-Ahead Log to speed up tests

The linked article suggests an interesting way to speed up tests of Rails + Postgres apps:

PostgreSQL allows the creation of “unlogged” tables, which do not record data in the PostgreSQL Write-Ahead Log. This can make the tables faster, but significantly increases the risk of data loss if the database crashes. As a result, this should not be used in production environments. If you would like all created tables to be unlogged in the test environment you can add the following to your...

open-next-failure: An alias to speed up test debugging

Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.

The following bash aliases helped me speed up that process:

alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json  | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"

There is a lot going on above but the gist...

Capistrano 3: Running a command on all servers

This Capistrano task runs a command on all servers.

bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'

Code

# lib/capistrano/tasks/app.rake

namespace :app do

  # Use e.g. to grep logs on all servers:
  #   b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log' 
  #
  # * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
  # * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
  # * To be able to process ...

Gitlab: How to cancel redundant pipelines

In the Gitlab settings the flag Auto-cancel redundant pipelines is enabled by default. This auto-cancels jobs that have the interruptible setting set to true (defaults to false e.g. to not cancel deploys by accident).

Consider to set the interruptible flag for test jobs to reduce the load on your runners like in the following example .gitlab-ci.yml:

rubocop:
  interruptible: true
  script:
    - 'bundle exec rubocop'

rspec:
  int...

Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)

General

A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch

This card deals with specifics concerning the use of searchkick.

Step 1: Make Opensearch available for Searchkick

In your Gemfile

# Search
gem 'searchkick'                   # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'

in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:

SEARCHKICK_CLIENT_T...

OpenAI TTS: How to generate audio samples with more than 4096 characters

OpenAI is currently limiting the Audio generating API endpoint to text bodies with a maximum of 4096 characters.
You can work around that limit by splitting the text into smaller fragments and stitch together the resulting mp3 files with a CLI tool like mp3wrap or ffmpeg.

Example Ruby Implementation

Usage

input_text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Mi eget mauris pharetra et ultrices neque."
output_mp3_path = Rails.root.join("tts/ipsum...

Livereload + esbuild

Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.

We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets and compiles to public/assets; if not change the paths below accordingly.

Basic idea

We will

  • use the guard-livereload gem as the livereload server (which send updates to the browser),
  • use the livereload-js npm package in the browser to con...

RSpec: Messages in Specific Order

tl;dr

You can use ordered to ensure that messages are received in a specific order.

Example

expect(ClassA).to receive(:call_first).ordered
expect(ClassB).to receive(:call_second).ordered
expect(ClassB).to receive(:call_third).ordered

#ordered supports further chaining

Example

RSpec: Composing a custom matcher from existing matchers

When you find similar groups of expect calls in your tests, you can improve readability by extracting the group into its own matcher. RSpec makes this easy by allowing matchers to call other matchers.

Example

The following test checks that two variables foo and bar (1) have no lowercase characters and (2) end with an exclamation mark:

expect(foo).to_not match(/[a-z]/)
expect(foo).to end_with('!')

expect(bar).to_not match(/[a-z]/)
expect(bar).to end_with('!')

We can extract the repeated matcher chains into a custom m...

Ruby: Using `sprintf` to replace a string at fixed named references

The sprintf method has a reference by name format option:

sprintf("%<foo>d : %<bar>f", { :foo => 1, :bar => 2 }) # => 1 : 2.000000
sprintf("%{foo}f", { :foo => 1 })                      # => "1f"

The format identifier %<id> stands for different data types to be formatted, such as %f for floats:

sprintf('%f', 1) # => 1.000000

Example:

This is quite useful to replace ...

Using Rationals to avoid rounding errors in calculations

Ruby has the class Rational which allows you to store exact fractions. Any calculation on these variables will now use fractional calculations internally, until you convert the result to another data type or do a calculation which requires an implicit conversion.

Example use case:

Lets say you want to store the conversion factor from MJ to kWh in a variable, which is 1/3.6. Using BigDecimals for this seems like a good idea, it usually helps with rounding errors over a float, but the...

A reasonable default CSP for Rails projects

Every modern Rails app should have a Content Security Policy enabled.

Very compatible default

The following "default" is a minimal policy that should

  • "just work" for almost all applications
  • give you most of the benefits of a CSP

In your config/initializers/content_security_policy.rb, set

Rails.application.config.content_security_policy do |policy|
  policy.object_src :none
  policy.script_src :unsafe_eval, :strict_dynamic, :https # Browsers with support for "'strict-dynamic'" will ignore "https:"
  po...

How to: Upgrade CarrierWave to 3.x

While upgrading CarrierWave from version 0.11.x to 3.x, we encountered some very nasty fails. Below are the basic changes you need to perform and some behavior you may eventually run into when upgrading your application. This aims to save you some time understanding what happens under the hood to possibly discover problems faster as digging deeply into CarrierWave code is very fun...

Whitelists and blacklists

The following focuses on extension allowlisting, but it is the exact same thing for content type allowlisting with the `content_ty...

Zeitwerk: How to collapse folders in Rails

All direct child directories of app are automatically added to the eager- and autoload paths. They do NOT create a module for namespacing. This is intuitive, since there normally is no module Model, or module Controller. If you want to add a new base directory, there's no additional config needed.

Example

app
├── controllers
├── helpers
├── inputs # No config needed 
├── mailers
├── models
├── uploaders # No config needed
├── util # No config needed
└── workers # No config needed

Sometimes it's handy to group files wit...

Solving "TypeError (nil can't be coerced into Integer)" in the Rails console / IRB

On the Rails console, assigning an object to a variable can lead to this strange error (without stacktrace):

irb > recipient = Recipient.find(123)
Traceback (most recent call last):
TypeError (nil can't be coerced into Integer)
irb > recipient
#<Recipient ...

The error is only in the output – the assignment is working. It only occurs when using the --nomultiline option, and thus [only with IRB 1.2.0+ and before Ruby 3](https://github.com/makandra/geordi/blob...

Ruby: `extend` extends the singleton class's inheritance chain

In the discussion of the difference between include and extend in Ruby, there is a misconception that extend would add methods to the singleton class of a ruby object as stated in many posts on this topic. But in fact, it is added to the ancestors chain of the singleton class! Even though it is technically not the same, practically this can be considered the same in most use cases.

Example

This means, that we are able to overwrite these methods or call the parent version with super depending in which order and in whi...

Heads Up: Selenium 4 uses a binary to determine the chromedriver

I recently stumbled over a problem that my feature tests broke in CI because of a mismatching chromedriver version.

In this specific project we have a fixed Chromium version in a Debian 12 environment instead of Chrome. The tests however used a recent chrome version instead.

$ chromedriver --version
ChromeDriver 117.0.5938.149 (e3344ddefa12e60436fa28c81cf207c1afb4d0a9-refs/branch-heads/5938@{#1539})
$ chromium --version
Chromium 117.0.5938.149 built on Debian 12.1, running on Debian 12.1

> WARN Selenium [:selenium_manager] The chromed...

Heads up: Quering array columns only matches equally sorted arrays

Given you have an array column like this:

create_table "users", force: :cascade do |t|
  t.integer "movie_ids", default: [], array: true
end

You might think that the following queries yield the same result:

User.where(movie_ids: [16, 17])
User.where(movie_ids: [17, 16])

Turn's out - they are not! They do care about array ordering more than I do.

To query for identical arrays independent of their order you have to either:

  1. Sort both the query and database content. If you're on Rails 7.1 you can use the new [`normal...

RSpec: Leverage the power of Capybara Finders and Matchers for view specs

View specs are a powerful tool to test several rendering paths by their cases instead of using a more costing feature spec. This is especially useful because they become quite convenient when used with Capybara::Node::Finders and Capybara::RSpecMatchers. This allows to wirte view unit specs as you can isolate specific part...

Spreewald, Cucumber: Selector for the nth element

The recommended additional setup of the spreewald gem, a useful set of cucumber steps, includes adding a file for defining custom selectors which can be used as prose within steps:

When I follow "Edit" within the controls section

Where the controls section can be any arbitrary defined css selector within selectors.rb


Often it can be useful to select the nth element of a specific selector. Luckily, this can ...

RSpec: Efficiently rerunning failed examples during development

Note

Don't use reruns as a mean to work around flaky tests. You should always try to fix those instead of rerunning them regularly.

Setup

Configure RSpec to persist the result of your test runs to a file. This is necessary to be able to rerun examples.
Add this to your spec/spec_helper.rb :

  config.example_status_persistence_file_path = 'spec/examples.txt'

Rerun all failed examples using --only-failures

bundle exec rspec --only-failures (or `...

Using Ruby's Method objects for inspecting methods

Do you remember finding where a method is defined?

I recently learned from a senior colleague that Method objects are quite useful within a debugging feast to find out the currently defined internals of methods, because they are either called within the current context or because you want to learn something about the API of the current objects.

Why is this useful?

This is especially useful since Ru...