Best practice: How to manage versions in a Gemfile

It most cases it's not necessary to add a version constraint next to your gems in the Gemfile. Since all versions are saved in the Gemfile.lock, everyone running bundle install will get exactly the same versions.

There are some exceptions, where you can consider adding a version constrain to the Gemfile:

  • You are not checking in the Gemfile.lock into the version control (not recommended)
  • A specific gem has a bug in a more recent version (adding a comment for the reason is highly recommended)
  • You want to ensure no one upgrade...

ActiveRecord: Passing an empty array into NOT IN will return no records

Caution when using .where to exclude records from a scope like this:

# Fragile - avoid
User.where("id NOT IN (?)", excluded_ids)

When the exclusion list is empty, you would expect this to return all records. However, this is not what happens:

# Broken example
User.where("id NOT IN (?)", []).to_sql
=>  SELECT `users`.* FROM `users` WHERE (id NOT IN (NULL))

Passing an empty exclusion list returns no records at all! See below for better implementations.

Rails 4+

Use the .not method to let Rails do the logic

`...

`simple_format` does not escape HTML tags

simple_format ignores Rails' XSS protection. Even when called with an unsafe string, HTML characters will not be escaped or stripped!

Instead simple_format calls sanitize on each of the generated paragraphs.

ActionView::Base.sanitized_allowed_tags
# => #<Set: {"small", "dfn", "sup", "sub", "pre", "blockquote", "ins", "ul", "var", "samp", "del", "h6", "h5", "h4", "h3", "h2", "h1", "span", "br", "hr", "em", "address", "img", "kbd", "tt", "a", "acrony...

Using rack-mini-profiler (with Unpoly)

Debugging performance issues in your Rails app can be a tough challenge.

To get more detailed insights consider using the rack-mini-profiler gem.

Setup with Unpoly

Add the following gems:

group :development do
  gem 'memory_profiler'
  gem 'rack-mini-profiler'
  gem 'stackprof'
end

Unpoly will interfere with the rack-mini-profiler widget, but configuring the following works okayish:

// rack-mini-profiler + unpoly
if (process...

Fixing flaky E2E tests

An end-to-end test (E2E test) is a script that remote-controls a web browser with tools like Selenium WebDriver. This card shows basic techniques for fixing a flaky E2E test suite that sometimes passes and sometimes fails.

Although many examples in this card use Ruby, Cucumber and Selenium, the techniques are applicable to all languages and testing tools.

Video available

▶️ Fixing Flaky E2E Tests

Why tests are flaky

Your tests probably look like this:

When I click ...

Best practices: Large data migrations from legacy systems

Migrating data from a legacy into a new system can be a surprisingly large undertaking. We have done this a few times. While there are significant differences from project to project, we do have a list of general suggestions.

Before you start, talk to someone who has done it before, and read the following hints:

Understand the old system

Before any technical considerations, you need to understand the old system as best as possible. If feasible, do not only look at its API, or database, or frontend, but let a user of the old system sho...

Testing ActiveRecord validations with RSpec

Validations should be covered by a model's spec.

This card shows how to test an individual validation. This is preferrable to save an entire record and see whether it is invalid.

Recipe for testing any validation

In general any validation test for an attribute :attribute_being_tested looks like this:

  1. Make a model instance (named record below)
  2. Run validations by saying record.validate
  3. Check if record.errors[:attribute_being_tested] contains the expected validation error
  4. Put the attribute into a valid state
  5. Run...

CarrierWave: Default Configuration and Suggested Changes

CarrierWave comes with a set of default configuration options which make sense in most cases. However, you should review these defaults and adjust for your project wherever necessary.

You will also find suggestions on what to change below.

Understanding the default configuration

Here is the current default config for version 2:

config.permissions = 0644
config.directory_permissions = 0755
config.storage_engines = {
  :f...

High-level data types with "composed_of"

I recently stumbled upon the Rails feature composed_of. One of our applications dealt with a lot of addresses and they were implemented as 7 separate columns in the DB and Rails models. This seemed like a perfect use case to try out this feature.

TLDR

The feature is still a VERY leaky abstraction. I ran into a lot of ugly edge cases.

It also doesn't solve the question of UI. We like to use simple_form. It's currently not possible to simply write `f...

Caching file properties with ActiveStorage Analyzers

When working with file uploads, we sometimes need to process intrinsic properties like the page count or page dimensions of PDF files. Retrieving those properties requires us to download (from S3 or GlusterFS) and parse the file, which is slow and resource-intensive.

Active Storage provides the metadata column on ActiveStorage::Blob to cache these values. You can either populate this column with ad-hoc metadata caching or with custom Analyzers.

Attachment...

How to disable logging for ActiveStorage's Disk Service routes

In development, we store files using ActiveStorage's disk service. This means that stored files are served by your Rails application, and every request to a file results in (at least!) one non-trivial log entry which can be annoying. Here is how to disable those log entries.

Example

Here is an example of what loading a single <img> in an example application writes to the Rails log.

Started GET "/rails/active_storage/blobs/redirect/..." for ::1 at ...
Processing by ActiveStorage::Blobs::RedirectController#show as SVG
  Parameter...

How to update a single gem conservatively

The problem

Calling bundle update GEMNAME will update a lot more gems than you think. E.g. when you do this:

bundle update cucumber-rails

... you might think this will only update cucumber-rails. But it actually updates cucumber-rails and all of its dependencies. This will explode in your face when one of these dependencies release a new version with breaking API changes. Which is all the time.

In the example above updating cucumber-rails will give you Capybara 2.0 (because capybara is a dependency of `cucumber-rail...

Be very careful with 301 and 308 redirects

Browsers support different types of redirects.

Be very careful with these status codes:

  • 301 Moved Permanently
  • 308 Permanent Redirect

Most browsers seem to cache these redirects forever, unless you set different Cache-Control headers. If you don't have any cache control headers, you can never change them without forcing users to empty their cache.

Note

By default Rails sends a ...

Preventing users from uploading malicious content

When you allow file uploads in your app, a user might upload content that hurts other users.

Our primary concern here is users uploading .html or .svg files that can run JavaScript and possibly hijack another user's session.

A secondary concern is that malicious users can upload executables (like an .exe or .scr file) and use your server to distribute it. However, modern operating systems usually warn before executing files that were downloaded from t...

Project management best practices: Technical debt summary

Maintaining larger projects makes it more difficult to balance refactoring and upgrade tasks according to its actual value. Consider to create and periodically maintain a summary, which helps you and your team in the decision which refactoring task should be taken next.

Template

Here is an template on how you might categorize your tasks:

| Technical debt | Estimated Efforts | Visible customer value| Customer value explained| Developer value|Developer value explained|
|-----------------------------|----------------|----------...

Ruby tempfiles

With the the Ruby Tempfile class you can create temporary files. Those files only stick around as long as you have a reference to those. If no more variable points to them, the GC may finalize the object at some point and the file will be removed from the file system. In other words: tempfiles are removed automatically. If you would then try to access the tempfile using its path (which you stored previously), you would get an error because the file no longer exists.

You can proactively unlink your tempfiles to delete them earlier...

CI Template for GitHub Actions

Usually our code lives on GitLab, therefore our documentation for CI testing is extensive in this environment. If you are tied to GitHub e.g. because your customer uses it, you may use the following GitHub Actions template for the CI integration. It includes jobs for rspec (parallelized using knapsack, unit + feature specs), rubocop, eslint, coverage and license_finder.

Note that GitHub does not allow the use of YAML anchors and aliases. You can instead use [compos...

Webpack(er): A primer

webpack is a very powerful asset bundler written in node.js to bundle (ES6) JavaScript modules, stylesheets, images, and other assets for consumption in browsers.

Webpacker is a wrapper around webpack that handles integration with Rails.

This is a short introduction.

Installation

If you haven't already, you need to install node.js and Yarn.

Then, put

gem 'webpacker', '~> 4.x' # check if 4.x is still cu...

A simpler default controller implementation

Rails has always included a scaffold script that generates a default controller implementation for you. Unfortunately that generated controller is unnecessarily verbose.

When we take over Rails projects from other teams, we often find that controllers are the unloved child, where annoying glue code has been paved over and over again, negotiating between request and model using implicit and convoluted protocols.

We prefer a different approach. We believe that among all the classes in a Rails project, controllers are some of the hardest to...

Deliver Paperclip attachments to authorized users only

When Paperclip attachments should only be downloadable for selected users, there are three ways to go.
The same applies to files in Carrierwave.

Variant: Deliver attachments through Rails

The first way is to store Paperclip attachments not in the default public/system, but in a private path like storage inside the current release. You should prefer this method when dealing with sensitive data.
...

Careful when using Time objects for generating ETags

You can use ETags to allow clients to use cached responses, if your application would send the same contents as before.

Besides what "actually" defines your response's contents, your application probably also considers "global" conditions, like which user is signed in:

class ApplicationController < ActionController::Base
  etag { current_user&.id }
  etag { current_user&.updated_at }
end

Under the hood, Rails generates an ETag header value like W/"f14ce3710a2a3187802cadc7e0c8ea99". In doing so, all objects from that etaggers...

Regex: Be careful when trying to match the start and/or end of a text

Ruby has two different ways to match the start and the end of a text:

  • ^ (Start of line) and $ (End of line)
  • \A (Start of string) and \z (End of string)

Most often you want to use \A and \z.

Here is a short example in which we want to validate the content type of a file attachment. Normally we would not expect content_type_1 to be a valid content type with the used regular expression image\/(jpeg|png). But as ^ and $ will match lines, it matches both content_type_1 and content_type_2. Using \A and \z will wo...

Compare library versions as "Gem::Version" instances, not as strings

Sometimes we have to write code that behaves differently based on the version of a specific gem or the Ruby Version itself. The version comparison can often be seen with simple string comparison like so.

# ❌ Not recommended
if Rails.version > '6.1.7.8' || RUBY_VERSION > '3.1.4'
  raise Error, 'please check if the monkey patch below is still needed'
end

If you are lucky, the version comparison above works by coincidence. But chances are that you are not: For example, Rails version 6.1.10.8 would not raise an error in the code ...

How to not die with ActionView::MissingTemplate when clients request weird formats

When HTTP clients make an request they can define which response formats they can process. They do it by adding a header to the HTTP request like this:

Accept: application/json

This means the client will only understand JSON responses.

When a Rails action is done, it will try to render a template for a format that the client understand. This means when all you are HTML templates, a request that only accepts application/json will raise an error:

An ActionView::MissingTemplate occurred in pages#foo:
  Missing templa...