Rails: Testing file downloads with request specs

tl;dr

Prefer request specs over end-to-end tests (Capybara) to joyfully test file downloads!

Why?

Testing file downloads via Capybara is not easy and results in slow and fragile tests. We tried different approaches and the best one is just okay.

Tests for file downloads via Capybara ...

  • ... are slow,
  • ... are fragile (breaks CI, breaks if Selenium driver changes, ...),
  • ... need workarounds for your specia...

Chaining Capybara matchers in RSpec

You can chain multiple Capybara matchers on the page or any element:

expect(page)
  .to have_content('Example Course')
  .and have_css('.course.active')
  .and have_button('Start')

When you chain multiple matchers using and, [Capybara will retry the entire chain](https://github.com/teamcapybara/capybara/blob/c0cbf4024c1abd48b0c22c2930e7b05af58ab284/lib/capybara/rspec/matc...

Be careful when checking scopes for blankness

Today I stumbled across a pretty harmless-looking query in our application which turned out to be pretty harmful and caused huge memory usage as well as downing our passenger workers by letting requests take up to 60 seconds. We had a method that received a scope and then checked, if the scope parameter was blank? and aborted the method execution in this case.

def foo(scope)
  return if scope.blank?
  
  # Use scope, e.g.
  scope.find(...)
end

We then called this method with an all scope: foo(Media::Document::Base.all). *...

Use <input type="number"> for numeric form fields

Any form fields where users enter numbers should be an <input type="number">.

Numeric inputs have several benefits over <input type="text">:

  • On mobile or tablet devices, number fields show a special virtual keyboard that shows mostly digit buttons.
  • Decimal values will be formatted using the user's language settings.
    For example, German users will see 1,23 for <input type="number" value="1.23">.
  • Values in the JavaScript API or when submitting forms to the server will always use a point as decimal separator (i.e. "1.23" eve...

How to configure Selenium WebDriver to not automatically close alerts or other browser dialogs

tl;dr

We recommend configuring Selenium's unhandled prompt behavior to "ignore".

When running tests in a real browser, we use Selenium. Each browser is controlled by a specific driver, e.g. Selenium::WebDriver::Chrome for Chrome.

There is one quirk to all drivers (at least those following the W3C webdriver spec) that can be impractical:
When any user prompt (like an alert) is encountered when trying to perform an action, they will [dismiss the dialog by default](https://w3c....

Best practices: Writing a Rails script (and how to test it)

A Rails script lives in lib/scripts and is run with bin/rails runner lib/scripts/.... They are a simple tool to perform some one-time actions on your Rails application. A Rails script has a few advantages over pasting some prepared code into a Rails console:

  • Version control
  • Part of the repository, so you can build on previous scripts for a similar task
  • You can have tests (see below)

Although not part of the application, your script is code and should adhere to the common quality standards (e.g. no spaghetti code). However, a script...

Ignore commits when git blaming

You can ignore certain commits when using git blame with the --ignore-revs-file option. This is handy to ignore large rubocop commits or big renamings in your project. You can add and commit a .git-blame-ignore-revs file in your project to track a list of commits that should be ignored.

# a list of commit shas
123...
456...

Use git blame with the --ignore-revs-file option and ignore the SHAs specified in .git-blame-ignore-revs.

git blame --ignore-revs-file .git-blame-ignore-revs

If you want to use this flag by def...

open-next-failure: An alias to speed up test debugging

Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.

The following bash aliases helped me speed up that process:

alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json  | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"

There is a lot going on above but the gist...

Gitlab: How to cancel redundant pipelines

In the Gitlab settings the flag Auto-cancel redundant pipelines is enabled by default. This auto-cancels jobs that have the interruptible setting set to true (defaults to false e.g. to not cancel deploys by accident).

Consider to set the interruptible flag for test jobs to reduce the load on your runners like in the following example .gitlab-ci.yml:

rubocop:
  interruptible: true
  script:
    - 'bundle exec rubocop'

rspec:
  int...

Rails: Pluck across associated tables

#pluck is commonly used as a performant way to retain single database values from an ActiveRecord::Relation

Book.pluck(:title, :price) #=> [["The Hobbit", "8.99"], ["The Alchemist", "7.89"]]

But #pluck can do more: you can query multiple tables as well!

Book.joins(:author).pluck("books.title, books.price, authors.name") #=> [["The Hobbit", "8.99", "J. R. R. Tolkien"], ["The Alchemist", "7.89", "Paulo Coelho"]]

Note the use of :author for the joins, and then authors for the pluck clause. The first corresp...

A reasonable default CSP for Rails projects

Every modern Rails app should have a Content Security Policy enabled.

Very compatible default

The following "default" is a minimal policy that should

  • "just work" for almost all applications
  • give you most of the benefits of a CSP

In your config/initializers/content_security_policy.rb, set

Rails.application.config.content_security_policy do |policy|
  policy.object_src :none
  policy.script_src :unsafe_eval, :strict_dynamic, :https # Browsers with support for "'strict-dynamic'" will ignore "https:"
  po...

Zeitwerk: How to collapse folders in Rails

All direct child directories of app are automatically added to the eager- and autoload paths. They do NOT create a module for namespacing. This is intuitive, since there normally is no module Model, or module Controller. If you want to add a new base directory, there's no additional config needed.

Example

app
├── controllers
├── helpers
├── inputs # No config needed 
├── mailers
├── models
├── uploaders # No config needed
├── util # No config needed
└── workers # No config needed

Sometimes it's handy to group files wit...

PSA: Be super careful with complex `eager_load` or `includes` queries

TLDR

Using .includes or .eager_load with 1-n associations is dangerous. Always use .preload instead.

Consider the following ActiveRecord query:

BlogPost.eager_load(
  :comments
  :attachments,
).to_a

(Let's assume we only have a couple of blog posts; if you use pagination the queries will be more complicated, but the point still stands.

Looks harmless enough? It is not.

The problem

ActiveRecord will rewrite this into a query using LEFT JOINs which looks something like this:

SELECT "blog_posts...

Heads up: Quering array columns only matches equally sorted arrays

Given you have an array column like this:

create_table "users", force: :cascade do |t|
  t.integer "movie_ids", default: [], array: true
end

You might think that the following queries yield the same result:

User.where(movie_ids: [16, 17])
User.where(movie_ids: [17, 16])

Turn's out - they are not! They do care about array ordering more than I do.

To query for identical arrays independent of their order you have to either:

  1. Sort both the query and database content. If you're on Rails 7.1 you can use the new [`normal...

How to make sure that manual deploy tasks (scheduled in Pivotal Tracker) are executed on deploy (with Capistrano)

We regularly have tasks that need to be performed around a deploy. Be it to notify operations about changed application behavior, be it to run a little oneline script after the deploy. Most database-related stuff can be handled by migrations, but every once in a while, we have tasks that are much easier to be performed manually.

Writing deploy tasks

Here is how we manage the deploy tasks themselves:

  • Deploy tasks are written inside the Pivotal Tracker story description, clearly marked (e.g. with a headline "Deploy task")
  • We disting...

Split your parallel tests by execution time and keep execution logs up to date

Both knapsack and parallel_tests have the option to split groups by historic execution time. The required logs for this might be outdated since you manually have to update and push them into your repository.

The following card includes an option how you can keep them consistently up to date with no extra effort locally and/or remotely.

How to always split by execution logs

Parallel Tests

The parallel_tests gem has the option flag `--group...

redirect_to and redirect

There are multiple ways to redirect URLs to a different URL in Rails, and they differ in small but important nuances.

Imagine you want to redirect the following url https://www.example.com/old_location?foo=bar to https://www.example.com/new_location?foo=bar.

Variant A

You can use ActionController::Redirecting#redirect_to in a controller action

class SomeController < ActionController::Base
  def old_location
    redirect_to(new_location_url(params.permit(:foo))) 
  end
end

This will:

  • It will redirect with a 302 st...

Do not pass params directly into url_for or URL helpers

Rails' url_for is useful for generating routes from a Hash, but can lead to an open redirect vulnerability.

Your application's generated route methods with a _url suffix are also affected because [they use url_for unter the hood](https://github.com/rails/rails...

Do not use "permit!" for params

Rails' Strong Parameters enable you to allow only specific values from request params to e.g. avoid mass assignment.

Usually, you say something like params.permit(:email, :password) and any extra parameters would be ignored, e.g. when calling to_h.
This is excellent and you should definitely use it.

What is permit! and why is it dangerous?

However, there is also params.permit! whic...

Minifying object properties in JavaScript files

An introduction to mangling

When you minify ("compress", "optimize") your JavaScript for production, the names of your functions and variables will be renamed for brevity. This process is often called mangling.

E.g. if this is your source code:

function function1() {
  function2()
}

After mangling it would look like this:

function a() {
  b()
}

Object properties are not mangled by default

Minfiers never mangle properties by default, as this can be an unsafe transformation. This leads to larger file sizes if...

CarrierWave: Processing images with libvips

When you write your next CarrierWave uploader, consider processing your images with libvips instead of ImageMagick.

Reasons for libvips

There are several upsides to using libvips over ImageMagick:

Balance your texts today with text-wrap: balance

So you have a heading that is just barely wider than the container it should fit into, and it wraps a single word to a new line and it's not really pretty?
Cry no more, for you can use text-wrap: balance today to fix that. At least in some browsers.

When browsers encounter a text-wrapping element with text-wrap: balance style, they will try breaking to a new line sooner, if it balances out the width of lines.

Without text-wrap: balance With text-wrap: balance
Image ![...

Checklist for Implementing Design

We have a long-standing checklist for merge requests. However, it hardly matches the intricate requirements for design. This checklist fills the gap.

Before starting implementing, look at all designs: are there components similar to yours? Have they already been implemented? Can you build on this prior art when implementing yours?

Checklist: I confirm my design implementation

  • has been tested manually by me
  • adheres to the code style of the project (e.g. BEM)
  • avoids "magic numbers" (don't say e.g. ...

Best practice: How to manage versions in a Gemfile

It most cases it's not necessary to add a version constraint next to your gems in the Gemfile. Since all versions are saved in the Gemfile.lock, everyone running bundle install will get exactly the same versions.

There are some exceptions, where you can consider adding a version constrain to the Gemfile:

  • You are not checking in the Gemfile.lock into the version control (not recommended)
  • A specific gem has a bug in a more recent version (adding a comment for the reason is highly recommended)
  • You want to ensure no one upgrade...