Five years of "Today I Learned" from Josh Branchaud

The linked GitHub repository is a bit like our "dev" cards deck, but groomed from a single person (Josh Branchaud). It includes an extensive list of over 900 TILs on many topics that might be interesting for most of us. (e.g. Ruby, Rails, Git, Unix..)

Ruby

Here is an excerpt of all the Ruby TILs that were new to me. I encourage you to take your time to skim over the original list as well!

Geordi: How to rerun failed features

Geordi's cucumber command has a --rerun option that reruns failing tests the given number of times. Usage:

geordi cucumber path/to/features --rerun=2
geordi cucumber path/to/features -r2

Background and how to rerun manually

Cucumber will save a file tmp/parallel_cucumber_failures.log containing the filenames and line number of the failed scenarios after a full test run. Normally you can say cucumber -p rerun (rerun is a profile defined by default in config/cucumber.yml) to rerun all failed scenarios.

Here are a few al...

Jasmine: Mocking ESM imports

In a Jasmine spec you want to spy on a function that is imported by the code under test. This card explores various methods to achieve this.

Example

We are going to use the same example to demonstrate the different approaches of mocking an imported function.

We have a module 'lib' that exports a function hello():

// lib.js

function hello() {
  console.log("hi world")
}

export hello

We have a second module 'client' that exports a function helloTwice(). All this does is call hello() ...

How to make changes to a Ruby gem (as a Rails developer)

At makandra, we've built a few gems over the years. Some of these are quite popular: spreewald (> 1M downloads), active_type (> 1M downloads), and geordi (> 200k downloads)

Developing a Ruby gem is different from developing Rails applications, with the biggest difference: there is no Rails. This means:

  • no defined structure (neither for code nor directories)
  • no autoloading of classes, i.e. you need to require all files yourself
  • no active_support niceties

Also, their scope...

Rails asset pipeline: Using ESNext without a transpiler

If your app does not need to support IE11, you can use most ES6 features without a build step. Just deliver your plain JavaScript without transpilation through Babel or TypeScript, and modern browsers will run them natively.

Features supported by all modern browsers include:

  • fat arrow functions (() => { expr })
  • let / const
  • class
  • async / await
  • Promises
  • Generators
  • Symbols
  • Rest arguments (...args)
  • Destructuring

You won't be able to use import and export, or use npm modules.

See this [ES6 compatibility mat...

Integrating or upgrading makandra-rubocop

Introduction

Most of the time it is a tedious task to apply a code style guide to an existing code base as there are likely to be a lot of conflicts. At makandra we are using makandra-rubocop to have code style checks. Here is some advice on how to add makandra-rubocop efficiently.

Note

RubyMine by default has a Rubocop inspection with rules that we don't always agree with. We recommend replacing this with makandra-rubocop or disabling the inspection.
...

Creating a Rails application in a single file

Greg Molnar has written a neat article about creating a single-file Rails app.
This is not meant for production use but can be useful to try things out, e.g. when hunting down a bug or embedding a Rails app into the tests of a gem.

What you do is basically:

  1. Put everything (gems, application config, database migrations, models, controllers) into a single .ru file, like app.ru.
  2. Run it via rackup app.ru. (Hint: if your file is called config.ru, you can just run `rac...

Managing chrome versions for selenium

Currently we often use geordi to run cucumber and rspec tests. Geordi takes care of installing a matching chromedriver for the installed google-chrome binary. The google-chrome binary is managed by apt.

Another approach is to use the Selenium Manager for installing and using the correct browser versions for you. Here is the setup you need for your integration tests:

Capybara.register_driver :chrome do |app|
  options = Sele...

How to: Self-hosted fonts via NPM packages

We usually ship applications that self-host webfonts to comply with GDPR.

Many popular web fonts are available as NPM packages provided by Fontsource.
We recommend using those instead of downloading and bundling font files yourself. (See below for a list of benefits.)

Usage

  1. Go to fontsource.org and search for the font you want to add (or a font that suits your application).
  2. Click the font card to vie...

Cucumber CI job quirks

Most of our CI pipelines don't use the --retry flag for Cucumber and instead build their own retry via the tmp/failing_features.txt file.

Benefits:

  • It's possible to only use -f pretty for the rerun.

Drawbacks:

  • MAJOR: With our current setup, when the main run fails without writing a tmp/failing_features.txt (e.g. due to a syntax error), the CI job will pass
  • MINOR: With our current setup, we lose the test coverage of the main run

A fix for the passing CI despite syntax error could look like this:

cucumber:
  # ...
  sc...

Maintaining custom application tasks in Rails

Here are some hints on best practices to maintain your tasks in larger projects.

Rake Tasks vs. Scripts

  • The Rails default is using rake tasks for your application tasks. These live in lib/tasks/*.
  • In case you want to avoid rake for your tasks and just use plain ruby scripts, consider lib/scripts/* as folder.

Keeping tasks slim

For readability and testing it's easier to keep your tasks slim. We suggest to use folders inside the tasks or scripts folder.

Example for a task:

The slim task lib/tasks/gitlab.rb:

Regular tasks for long-running projects

When projects run for many years, they require special regular maintenance to stay fresh. This kind of maintenance is usually not necessary for small or quick projects, but required to keep long-running projects from getting stale.

You should be able to fit this into a regular development block.

Quarterly

Check which libraries need updating

As time goes by, libraries outdate. Check your software components and decide if any of it needs an update. Your main components (e.g. Ruby, Rails, Unpoly) should always be reasonably up to da...

Knapsack: Rerun a flaky test locally

Knapsack allows you to rerun a specific job locally. This is helpful to run specs in the exactly same order like in the CI.

Example for running rspec 3/8 with a seed output of 20689:

CI_NODE_INDEX=2 CI_NODE_TOTAL=8 bundle exec rake "knapsack:rspec[--seed=20689]"

*Note: the environment variable ...

Automated "git bisect" will make your day

So you're hunting down a regression (or just a bug) and want to use git bisect to find out when it was introduced? Smart kid.
If you have a shell command ready to reveal if your current state is good or bad, you can have git do most of the work for you.

Using git bisect run <your command> you can tell git that your command will reveal the issue; git on the other hand will use the return value of that call to decide if the state is good or bad.
...

Whenever requires you to set the application attribute in the Capistrano config

Whenever requires you to set the application attribute in your Capistrano configuration. Otherwise your cronjobs are created multiple times.

Example entry in config/deploy.rb:

set :application, 'some-app' # allows "set :whenever_identifier, ->{ "#{fetch(:application)}_#{fetch(:stage)}" }" to work as expected

Good

Then the crontab -l output will look like this:

# Begin Whenever generated tasks for: som...

Generating an Entity Relationship Diagram for your Rails application

This card explains how to generate an entity relationship diagram for your Rails application.
We also show how to limit your ERD to a subset of models, e.g. models inside a namespace.

Generating a full ERD

Option A: RubyMine

  1. Right-click anywhere in your project tree
  2. In the context menu, find the "Diagrams" menu item at/near the bottom
  3. Inside, choose "Show diagram" → "Rails Model Dependency Diagram"
  4. A new tab will open with the diagram inside. You can modify it there, and export it as an image.

Option B: Use rails-e...

whenever: Installing cron jobs only for a given Rails environment or Capistrano stage

We use the whenever gem to automatically update the crontab of the servers we deploy to. By default, whenever will update all servers with a matching role (we use the :cron role ).

This card describes how to install some tasks only for a given Rails environment or for a given Capistrano stage ("deployment target").

Installing jobs only for a given Rails environment
-----------------------------------...

Rails I18n fallback locales

When you need to create a locale for a language variant (like Austrian for German), you probably don't want to duplicate your entire de.yml file only to change a few minor exceptions for our Austrian friends.

Luckily, the I18n gem used by Rails has a fallback feature where you can make one locale file fall back to another if no translation is available.

In the example above you would have a config/locales/de_DE.yml:

de_DE:
  # hundreds of translations here

... and another...

Auto-generating plain-text bodies for HTML e-mails in Rails apps

When building an application that sends e-mails to users, you want to avoid those e-mails from being classified as spam. Most obvious scoring issues will not be relevant to you because you are not a spammer.

However, your application must do one thing by itself: When sending HTML e-mails, you should include a plain-text body or tools like SpamAssassin will apply a significant score penalty. Here is how to do that automatically.

  1. Add premailer-rails to your Gemfile and bundle.
  2. Done! ...

Webpacker: Configuring browser compatibility

Webpacker uses Babel and Webpack to transpile modern JavaScript down to EcmaScript 5. Depending on what browser a project needs to support, the final Webpack output needs to be different. E.g. when we need to support IE11 we can rely on fewer JavaScript features. Hence our output will be more verbose than when we only need support modern browsers.

Rails 5.1+ projects often use Webpacker to preconfigure the Webpack pipeline for us. The default configuration works something like this:

  1. Webpack checks w...

Split your parallel tests by execution time and keep execution logs up to date

Both knapsack and parallel_tests have the option to split groups by historic execution time. The required logs for this might be outdated since you manually have to update and push them into your repository.

The following card includes an option how you can keep them consistently up to date with no extra effort locally and/or remotely.

How to always split by execution logs

Parallel Tests

The parallel_tests gem has the option flag `--group...

Temporary solution for connection errors with rubygems

The problem

If you're experiencing that your bundle install command fails with an error message like this, rubygems.org might have issues with their ipv6 connectivity:

$ bundle install
Fetching source index from https://rubygems.org/

Retrying fetcher due to error (2/4): Bundler::HTTPError Could not fetch specs from https://rubygems.org/ due to underlying error <timed out (https://rubygems.org/specs.4.8.gz)>

The (a little bit dirty) possible solution

If that's actually the case, then you can try to deprioritize the ipv...

How to search through logs on staging or production environments

We generally use multiple application servers (at least two) and you have to search on all of them if you don't know which one handled the request you are looking for.

Rails application logs usually live in /var/www/<project-environment-name>/shared/log.
Web server logs usually live in /var/www/<project-environment-name>/log.

Searching through single logs with grep / zgrep

You can use grep in this directory to only search the latest logs or zgrep to also search older (already zipped) logs. zgrep is used just like grep ...

Debugging Capistrano

Capistrano 3 has a doctor task that will print information about

  • Environment: Ruby, Rubygems and Bundler versions
  • List of Capistrano gems and whether an update is available
  • All config variables and their values
  • Capistrano server config
$ bundle exec cap staging doctor