CI Template for GitHub Actions
Usually our code lives on GitLab, therefore our documentation for CI testing is extensive in this environment. If you are tied to GitHub e.g. because your customer uses it, you may use the following GitHub Actions template for the CI integration. It includes jobs for rspec (parallelized using knapsack, unit + feature specs), rubocop, eslint, coverage and license_finder.
Note that GitHub does not allow the use of YAML anchors and aliases. You can instead use [compos...
Firefox ESR Release Calendar
The linked table shows the support lifecycle for Firefox Extended Support Releases (ESR) which we sometimes need to support for enterprise customers.
The ESR cadence works something like this:
- Firefox ESR freezes the then-current Firefox version for a year.
- During this year Mozilla backports security patches to the current ESR, but does not add features.
- Two subsequent ESR releases overlap for three months. This way enterprises have a quarter to test the new version and migrate their clients.
How to configure Selenium WebDriver to not automatically close alerts or other browser dialogs
tl;dr
We recommend configuring Selenium's unhandled prompt behavior to
{ default: 'ignore' }with the monkey patch below.
When running tests in a real browser, we use Selenium. Each browser is controlled by a specific driver, e.g. Selenium::WebDriver::Chrome for Chrome.
There is one quirk to all drivers (at least those following the W3C webdriver spec) that can be impractical:
When any user prompt (like an alert) is encountered when trying to perform an action, they will [...
Best practices: Writing a Rails script (and how to test it)
A Rails script lives in lib/scripts and is run with bin/rails runner lib/scripts/.... They are a simple tool to perform some one-time actions on your Rails application. A Rails script has a few advantages over pasting some prepared code into a Rails console:
- Version control
- Part of the repository, so you can build on previous scripts for a similar task
- You can have tests (see below)
Although not part of the application, your script is code and should adhere to the common quality standards (e.g. no spaghetti code). However, a scri...
Bash: How to count and sort requests by IP from the access logs
Example
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.41 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
Goal
Count and sort the number of requests for a single IP address.
Bash Command
awk '{ print $1}' test.log | sort...
Bash: How to grep logs for a pattern and expand it to the full request
Example
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [53a240c1-489e-4936-bbeb-d6f77284cf38] nope
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- ...
Rails: Testing the number of database queries
There are a few tools to combat the dreaded n+1 queries. The bullet gem notifies you of missing eager-loading, and also if there is too much eager-loading. strict_loading in Rails 6.1+ forces developers to explicitly load associations on individual records, for a single association, for an entire model, or globally for all models.
But you can also actually **write spe...
Element.animate() to animate single elements with given keyframes
Today I learned that you can animate HTML elements using the Web Animation API's method .animate(keyframes, options) (which seems to be Baseline for all browsers since 2022).
const fadeIn = [{ opacity: 0 }, { opacity: 1 }] // this is a keyframe example
const container = document.querySelector('.animate-me')
const animation = container.animate(fadeIn, 2000) // I am just using the animation duration as option here but it can also be given an object
// the animation object can be used for things like querying timings or stat...
SASS: Adding, removing and converting units
Adding a unit
Multiply by 1x the unit:
$number = 13
$length = $number * 1px // => 13px
Removing a unit
Divide by 1x the unit:
$length = 13px
$number = $length / 1px // => 13
Converting a unit
the result of an addition or subtraction between two numbers of different units is expressed in the first member’s unit
Thus, to convert a number, add it to 0 of the desired unit:
$duration: .21s
$duration-in-milliseconds: 0ms + $duration // => 210ms
An example is storing a transition duration as CS...
When loading Yaml contents in Ruby, use the :freeze argument to deep-freeze everything
Ruby methods which load from a Yaml file, like YAML.safe_load or YAML.safe_load_file, support passing freeze: true to deep-freeze the entire contents from the Yaml file.
This is available by default on Ruby 3.0 and newer. On older Rubies, you can install psych 3.2.0 or newer for :freeze support.
As an example, consider the following Yaml file:
---
message:
- hello
- universe
foo:
bar:
baz: "example"
We can now load it as usual, but pass freeze: true.
>> test = YAML.safe_load_file('example.yml', fre...
Creating a self-signed certificate for local HTTPS development
Your development server is usually running on an insecure HTTP connection which is perfectly fine for development.
If you need your local dev server to be accessible via HTTPS for some reason, you need both a certificate and its key. For a local hostname, you need to create those yourself.
This card explains how to do that and how to make your browser trust the certificate so it does not show warnings for your own certificate.
Easy: self-signed certificate
To just create a certificate for localhost, you can use the following command....
Maintaining custom application tasks in Rails
Here are some hints on best practices to maintain your tasks in larger projects.
Rake Tasks vs. Scripts
- The Rails default is using rake tasks for your application tasks. These live in
lib/tasks/*. - In case you want to avoid rake for your tasks and just use plain ruby scripts, consider
lib/scripts/*as folder.
Keeping tasks slim
For readability and testing it's easier to keep your tasks slim. We suggest to use folders inside the tasks or scripts folder.
Example for a task:
The slim task lib/tasks/gitlab.rb:
Rails: Using custom config files with the config_for method
You can use the config.x configuration in combination with config_for to configure global settings for your Rails 4.2+ application.
Example
In your config/application.rb assign the settings from e.g. config/settings.yml as follows:
module FooApplication
class Application < Rails::Application
config.x.settings = config_for(:settings)
end
end
The config/settings.yml might look as follows:
shared: &shared
email: info@example.com
...
Disable PostgreSQL's Write-Ahead Log to speed up tests
The linked article suggests an interesting way to speed up tests of Rails + Postgres apps:
PostgreSQL allows the creation of “unlogged” tables, which do not record data in the PostgreSQL Write-Ahead Log. This can make the tables faster, but significantly increases the risk of data loss if the database crashes. As a result, this should not be used in production environments. If you would like all created tables to be unlogged in the test environment you can add the following to your...
open-next-failure: An alias to speed up test debugging
Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.
The following bash aliases helped me speed up that process:
alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"
There is a lot going on above but the gist...
Capistrano 3: Running a command on all servers
This Capistrano task runs a command on all servers.
bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
Code
# lib/capistrano/tasks/app.rake
namespace :app do
# Use e.g. to grep logs on all servers:
# b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
#
# * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
# * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
# * To be able to process ...
Gitlab: How to cancel redundant pipelines
In the Gitlab settings the flag Auto-cancel redundant pipelines is enabled by default. This auto-cancels jobs that have the interruptible setting set to true (defaults to false e.g. to not cancel deploys by accident).
Consider to set the interruptible flag for test jobs to reduce the load on your runners like in the following example .gitlab-ci.yml:
rubocop:
interruptible: true
script:
- 'bundle exec rubocop'
rspec:
int...
Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)
General
A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch
This card deals with specifics concerning the use of searchkick.
Step 1: Make Opensearch available for Searchkick
In your Gemfile
# Search
gem 'searchkick' # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'
in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:
SEARCHKICK_CLIENT_T...
OpenAI TTS: How to generate audio samples with more than 4096 characters
OpenAI is currently limiting the Audio generating API endpoint to text bodies with a maximum of 4096 characters.
You can work around that limit by splitting the text into smaller fragments and stitch together the resulting mp3 files with a CLI tool like mp3wrap or ffmpeg.
Example Ruby Implementation
Usage
input_text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Mi eget mauris pharetra et ultrices neque."
output_mp3_path = Rails.root.join("tts/ipsum...
Livereload + esbuild
Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.
We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets and compiles to public/assets; if not change the paths below accordingly.
Basic idea
We will
- use the
guard-livereloadgem as the livereload server (which send updates to the browser), - use the
livereload-jsnpm package in the browser to con...
RSpec: Messages in Specific Order
tl;dr
You can use
orderedto ensure that messages are received in a specific order.
Example
expect(ClassA).to receive(:call_first).ordered
expect(ClassB).to receive(:call_second).ordered
expect(ClassB).to receive(:call_third).ordered
#ordered supports further chaining
RSpec: Composing a custom matcher from existing matchers
When you find similar groups of expect calls in your tests, you can improve readability by extracting the group into its own matcher. RSpec makes this easy by allowing matchers to call other matchers.
Example
The following test checks that two variables foo and bar (1) have no lowercase characters and (2) end with an exclamation mark:
expect(foo).to_not match(/[a-z]/)
expect(foo).to end_with('!')
expect(bar).to_not match(/[a-z]/)
expect(bar).to end_with('!')
We can extract the repeated matcher chains into a custom m...
Ruby: Using `sprintf` to replace a string at fixed named references
The sprintf method has a reference by name format option:
sprintf("%<foo>d : %<bar>f", { :foo => 1, :bar => 2 }) # => 1 : 2.000000
sprintf("%{foo}f", { :foo => 1 }) # => "1f"
The format identifier %<id> stands for different data types to be formatted, such as %f for floats:
sprintf('%f', 1) # => 1.000000
Example:
This is quite useful to replace ...
Using Rationals to avoid rounding errors in calculations
Ruby has the class Rational which allows you to store exact fractions. Any calculation on these variables will now use fractional calculations internally, until you convert the result to another data type or do a calculation which requires an implicit conversion.
Example use case:
Lets say you want to store the conversion factor from MJ to kWh in a variable, which is 1/3.6. Using BigDecimals for this seems like a good idea, it usually helps with rounding errors over a float, but the...