CI Template for GitHub Actions

Usually our code lives on GitLab, therefore our documentation for CI testing is extensive in this environment. If you are tied to GitHub e.g. because your customer uses it, you may use the following GitHub Actions template for the CI integration. It includes jobs for rspec (parallelized using knapsack, unit + feature specs), rubocop, eslint, coverage and license_finder.

Note that GitHub does not allow the use of YAML anchors and aliases. You can instead use [compos...

Firefox ESR Release Calendar

The linked table shows the support lifecycle for Firefox Extended Support Releases (ESR) which we sometimes need to support for enterprise customers.

The ESR cadence works something like this:

  • Firefox ESR freezes the then-current Firefox version for a year.
  • During this year Mozilla backports security patches to the current ESR, but does not add features.
  • Two subsequent ESR releases overlap for three months. This way enterprises have a quarter to test the new version and migrate their clients.

How to configure Selenium WebDriver to not automatically close alerts or other browser dialogs

tl;dr

We recommend configuring Selenium's unhandled prompt behavior to { default: 'ignore' } with the monkey patch below.

When running tests in a real browser, we use Selenium. Each browser is controlled by a specific driver, e.g. Selenium::WebDriver::Chrome for Chrome.

There is one quirk to all drivers (at least those following the W3C webdriver spec) that can be impractical:
When any user prompt (like an alert) is encountered when trying to perform an action, they will [...

Best practices: Writing a Rails script (and how to test it)

A Rails script lives in lib/scripts and is run with bin/rails runner lib/scripts/.... They are a simple tool to perform some one-time actions on your Rails application. A Rails script has a few advantages over pasting some prepared code into a Rails console:

  • Version control
  • Part of the repository, so you can build on previous scripts for a similar task
  • You can have tests (see below)

Although not part of the application, your script is code and should adhere to the common quality standards (e.g. no spaghetti code). However, a scri...

Bash: How to count and sort requests by IP from the access logs

Example

87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.41 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"

Goal

Count and sort the number of requests for a single IP address.

Bash Command

awk '{ print $1}' test.log | sort...

Bash: How to grep logs for a pattern and expand it to the full request

Example

I, [2024-01-21T06:22:17.484221 #2698200]  INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] test
I, [2024-01-21T06:22:17.484221 #2698200]  INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] more
I, [2024-01-21T06:22:17.484221 #2698200]  INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] test
I, [2024-01-21T06:22:17.484221 #2698200]  INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] more
I, [2024-01-21T06:22:17.484221 #2698200]  INFO -- : [53a240c1-489e-4936-bbeb-d6f77284cf38] nope
I, [2024-01-21T06:22:17.484221 #2698200]  INFO -- ...

Rails: Testing the number of database queries

There are a few tools to combat the dreaded n+1 queries. The bullet gem notifies you of missing eager-loading, and also if there is too much eager-loading. strict_loading in Rails 6.1+ forces developers to explicitly load associations on individual records, for a single association, for an entire model, or globally for all models.

But you can also actually **write spe...

Element.animate() to animate single elements with given keyframes

Today I learned that you can animate HTML elements using the Web Animation API's method .animate(keyframes, options) (which seems to be Baseline for all browsers since 2022).

const fadeIn = [{ opacity: 0 }, { opacity: 1 }] // this is a keyframe example

const container = document.querySelector('.animate-me')

const animation = container.animate(fadeIn, 2000) // I am just using the animation duration as option here but it can also be given an object

// the animation object can be used for things like querying timings or stat...

SASS: Adding, removing and converting units

Adding a unit

Multiply by 1x the unit:

$number = 13
$length = $number * 1px // => 13px

Removing a unit

Divide by 1x the unit:

$length = 13px
$number = $length / 1px // => 13

Converting a unit

the result of an addition or subtraction between two numbers of different units is expressed in the first member’s unit

Thus, to convert a number, add it to 0 of the desired unit:

$duration: .21s
$duration-in-milliseconds: 0ms + $duration // => 210ms

An example is storing a transition duration as CS...

When loading Yaml contents in Ruby, use the :freeze argument to deep-freeze everything

Ruby methods which load from a Yaml file, like YAML.safe_load or YAML.safe_load_file, support passing freeze: true to deep-freeze the entire contents from the Yaml file.
This is available by default on Ruby 3.0 and newer. On older Rubies, you can install psych 3.2.0 or newer for :freeze support.

As an example, consider the following Yaml file:

---
message:
  - hello
  - universe
foo:
  bar:
    baz: "example"

We can now load it as usual, but pass freeze: true.

>> test = YAML.safe_load_file('example.yml', fre...

Creating a self-signed certificate for local HTTPS development

Your development server is usually running on an insecure HTTP connection which is perfectly fine for development.

If you need your local dev server to be accessible via HTTPS for some reason, you need both a certificate and its key. For a local hostname, you need to create those yourself.
This card explains how to do that and how to make your browser trust the certificate so it does not show warnings for your own certificate.

Easy: self-signed certificate

To just create a certificate for localhost, you can use the following command....

Maintaining custom application tasks in Rails

Here are some hints on best practices to maintain your tasks in larger projects.

Rake Tasks vs. Scripts

  • The Rails default is using rake tasks for your application tasks. These live in lib/tasks/*.
  • In case you want to avoid rake for your tasks and just use plain ruby scripts, consider lib/scripts/* as folder.

Keeping tasks slim

For readability and testing it's easier to keep your tasks slim. We suggest to use folders inside the tasks or scripts folder.

Example for a task:

The slim task lib/tasks/gitlab.rb:

Rails: Using custom config files with the config_for method

You can use the config.x configuration in combination with config_for to configure global settings for your Rails 4.2+ application.

Example

In your config/application.rb assign the settings from e.g. config/settings.yml as follows:

module FooApplication
  class Application < Rails::Application
    config.x.settings = config_for(:settings)
  end
end

The config/settings.yml might look as follows:

shared: &shared
  email: info@example.com
...

Disable PostgreSQL's Write-Ahead Log to speed up tests

The linked article suggests an interesting way to speed up tests of Rails + Postgres apps:

PostgreSQL allows the creation of “unlogged” tables, which do not record data in the PostgreSQL Write-Ahead Log. This can make the tables faster, but significantly increases the risk of data loss if the database crashes. As a result, this should not be used in production environments. If you would like all created tables to be unlogged in the test environment you can add the following to your...

open-next-failure: An alias to speed up test debugging

Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.

The following bash aliases helped me speed up that process:

alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json  | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"

There is a lot going on above but the gist...

Capistrano 3: Running a command on all servers

This Capistrano task runs a command on all servers.

bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'

Code

# lib/capistrano/tasks/app.rake

namespace :app do

  # Use e.g. to grep logs on all servers:
  #   b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log' 
  #
  # * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
  # * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
  # * To be able to process ...

Gitlab: How to cancel redundant pipelines

In the Gitlab settings the flag Auto-cancel redundant pipelines is enabled by default. This auto-cancels jobs that have the interruptible setting set to true (defaults to false e.g. to not cancel deploys by accident).

Consider to set the interruptible flag for test jobs to reduce the load on your runners like in the following example .gitlab-ci.yml:

rubocop:
  interruptible: true
  script:
    - 'bundle exec rubocop'

rspec:
  int...

Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)

General

A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch

This card deals with specifics concerning the use of searchkick.

Step 1: Make Opensearch available for Searchkick

In your Gemfile

# Search
gem 'searchkick'                   # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'

in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:

SEARCHKICK_CLIENT_T...

OpenAI TTS: How to generate audio samples with more than 4096 characters

OpenAI is currently limiting the Audio generating API endpoint to text bodies with a maximum of 4096 characters.
You can work around that limit by splitting the text into smaller fragments and stitch together the resulting mp3 files with a CLI tool like mp3wrap or ffmpeg.

Example Ruby Implementation

Usage

input_text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Mi eget mauris pharetra et ultrices neque."
output_mp3_path = Rails.root.join("tts/ipsum...

Livereload + esbuild

Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.

We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets and compiles to public/assets; if not change the paths below accordingly.

Basic idea

We will

  • use the guard-livereload gem as the livereload server (which send updates to the browser),
  • use the livereload-js npm package in the browser to con...

RSpec: Messages in Specific Order

tl;dr

You can use ordered to ensure that messages are received in a specific order.

Example

expect(ClassA).to receive(:call_first).ordered
expect(ClassB).to receive(:call_second).ordered
expect(ClassB).to receive(:call_third).ordered

#ordered supports further chaining

Example

RSpec: Composing a custom matcher from existing matchers

When you find similar groups of expect calls in your tests, you can improve readability by extracting the group into its own matcher. RSpec makes this easy by allowing matchers to call other matchers.

Example

The following test checks that two variables foo and bar (1) have no lowercase characters and (2) end with an exclamation mark:

expect(foo).to_not match(/[a-z]/)
expect(foo).to end_with('!')

expect(bar).to_not match(/[a-z]/)
expect(bar).to end_with('!')

We can extract the repeated matcher chains into a custom m...

Ruby: Using `sprintf` to replace a string at fixed named references

The sprintf method has a reference by name format option:

sprintf("%<foo>d : %<bar>f", { :foo => 1, :bar => 2 }) # => 1 : 2.000000
sprintf("%{foo}f", { :foo => 1 })                      # => "1f"

The format identifier %<id> stands for different data types to be formatted, such as %f for floats:

sprintf('%f', 1) # => 1.000000

Example:

This is quite useful to replace ...

Using Rationals to avoid rounding errors in calculations

Ruby has the class Rational which allows you to store exact fractions. Any calculation on these variables will now use fractional calculations internally, until you convert the result to another data type or do a calculation which requires an implicit conversion.

Example use case:

Lets say you want to store the conversion factor from MJ to kWh in a variable, which is 1/3.6. Using BigDecimals for this seems like a good idea, it usually helps with rounding errors over a float, but the...