How to let passenger restart after deployment with capistrano

Phusion Passenger changed the way how it gets restarted several times. Through the project's history, these all were valid:

  • touch tmp/restart.txt
  • sudo passenger-config restart-app /path/to/app
  • passenger-config restart-app /path/to/app

You should not need to know which one to use. Instead, the capistrano-passenger gem will choose the appropriate restart mechanism automatically based on your installed the passenger version.

Installation

  1. Add to your Gemfile:

    gem 'capistr...
    

SAML Single Logout (SLO)

There are two ways a logout in SAML can happen: Service Provider (SP) initiated and Identity Provider (IDP) initiated logout. I'll explain how to implement both flows with devise_saml_authenticatable.

Note

SAML also supports a SOAP and an Artifact binding to do this. This guide only refers to POST and Redirect bindings. devise_saml_authenticatable does not support SOAP and Artifact bindings.

SP initiated logout (using the Redirect Binding)

When the user clicks on Logout within the app, the app can trigger...

Using ActiveRecord with threads might use more database connections than you think

Database connections are not thread-safe. That's why ActiveRecord uses a separate database connection for each thread.

For instance, the following code uses 3 database connections:

3.times do
  Thread.new do
    User.first # first database access makes a new connection
  end
end

These three connections will remain connected to the database server after the threads terminate. This only affects threads that use ActiveRecord.

You can rely on Rails' various clean-up mechanisms to release connections, as outlined below. This may...

Common mistakes when storing file uploads with Rails

1. Saving files to a directory that is not shared between deploys or servers

If you save your uploads to a made up directory like "RAILS_ROOT/uploads", this directory goes away after every deploy (since every release gets a new). Also this directory is not shared between multiple application servers, so your uploads are randomly saved to one local filesystem or another. Fixing this afterwards is a lot of fun.

Only two folders are, by default, shared between our application servers and deployments: "RAILS_ROOT/storage" and `"RAILS...

Be careful to use correct HTTP status codes for maintenance pages

When your public-facing application has a longer downtime for server maintenance or long migrations, it's nice to setup a maintenance page to inform your users.

When delivering the maintenance page, be very careful to send the correct HTTP status code. Sending the wrong status code might get you kicked out of Google, or undo years of SEO work.

Popular footguns

Here are some ways to shoot yourself in the foot during maintenance:

  • If all your routes send a "200 OK" with a HTML body "We're b...

Capybara: Waiting for pending AJAX requests after a test

When ending a Selenium test Capybara resets the browser state by closing the tab, clearing cookies, localStorage, etc.

It may be a good idea to wait for all in-flight AJAX requests to finish before ending a scenario:

  • You may have client-side JavaScript that freaks out when the tab closure kills their pending requests. If that JavaScript opens an error alert or spams errors to the console, your test may fail after the last step.
  • With unlucky timing the server may receive an AJAX request as the browser tab closes, causing a connection ...

E-mail deliverability

When your application is open for public sign up and sends out transactional e-mails to a large number of users, e-mail deliverability becomes an issue.

E-mail providers work hard to eliminate spam and have put in place relatively tight checks what kinds of emails they will accept, and from whom. To that end we use tools like mail-tester.com to make our mails as acceptable as possible. Unfortunately, there will always be providers that reject our e-mails for some reason or other, sometimes outside of our control.

For exa...

Differences between transactions and locking

Web applications can be used by multiple users at the same time. A typical application server like Passenger has multiple worker processes for a single app. In a distributed deployment setup like we use at makandra you will even have multiple application servers, each with their own worker pool.

This means that your code needs to deal with concurrent data access. The two main tools we use to cope with concurrency are database transactions and distributed locks. These two are not interchangeable. You ca...

Using the Truemail gem to validate e-mail addresses

The Truemail gem (not to be confused with truemail.io) allows validating email addresses, e.g. when users enter them into a sign-up form. It runs inside your application and does not depend on an external SaaS service.

Truemail supports different validation "layers":

  1. Regex validation: if the given address is syntactically valid
  2. DNS validation (called MX validation): if the given domain exists and can receive email
  3. SMTP validation: connects to the host received from DNS and starts a test d...

Working with or without time zones in Rails applications

Rails supports time zones, but there are several pitfalls. Most importantly because Time.now and Time.current are completely different things and code from gems might use one or the other.

Especially configuring an application that cares only about one time zone is a bit tricky.

The following was tested on Rails 5.1 but should apply to Rails 4.2 as well.

Using only local time

Your life will be easier if your application does not need to support time zones. Disable them like this:

config.time_zone = 'Berlin' # Your local ...

Rails: Keeping structure.sql stable between developers

Why Rails has multiple schema formats

When you run migrations, Rails will write your current database schema into db/schema.rb. This file allows to reset the database schema without running migrations, by running rails db:schema:load.

The schema.rb DSL can serialize most common schema properties like tables, columns or indexes. It cannot serialize more advanced database features, like views, procedures, triggers or custom ditionaries. In these cases you must switch to a SQL based schema format:

# in application.rb
config.a...

How to Work With Time Zones in Rails

When dealing with time zones in Rails, there is one key fact to keep in mind:

Rails has configurable time zones, while
Ruby is always in the server's time zone

Thus, using Ruby's time API will give you wrong results for different time zones.

"Without" time zones

You can not actually disable time zones, because their existence is a fact. You can, however, tell Rails the only single time zone you'll need is the server's.

config.time_zone = "Berlin" # Local time zone
config.active_record.default_timezone = :loca...

Don't assert exceptions in feature specs

As we are slowly switching from Cucumber scenarios to RSpec feature specs, you might be tempted to write assertions like this one:

feature 'authorization for cards management' do
  let(:guest_user) { create(:user, :guest) }

  scenario "rejects guest users from adding new cards", js: true do
    sign_in guest_user

    expect { visit new_cards_path }.to raise_error(Consul::Powerless)
  end
end

While this might work under certain circumstances¹, there is a good chance you'll see two exceptions when running this single spec:

  • ...

Livereload + esbuild

Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.

We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets and compiles to public/assets; if not change the paths below accordingly.

Basic idea

We will

  • use the guard-livereload gem as the livereload server (which send updates to the browser),
  • use the livereload-js npm package in the browser to con...

Configuring Webpacker deployments with Capistrano

When deploying a Rails application that is using Webpacker and Capistrano, there are a few configuration tweaks that optimize the experience.

Using capistrano-rails

capistrano-rails is a Gem that adds Rails specifics to Capistrano, i.e. support for Bundler, assets, and migrations. While it is designed for Asset Pipeline (Sprockets) assets, it can easily be configured for Webpacker. This brings these features to the Webpacker world:

  • Automatic removal of expired assets
  • Manifest backups

HTML5: disabled vs. readonly form fields

Form fields can be rendered as noneditable by setting the disabled or the readonly attribute. Be aware of the differences:

disabled fields

  • don’t post to the server
  • don’t get focus
  • are skipped while tab navigation
  • available for button, fieldset, input, select, textarea, command, keygen, optgroup, option

Browser specific behavior:

  • IE 11: text inputs that are descendants of a disabled fieldset appear disabled but the user can still interact with them
  • Firefox: selecting text in a disabled text field is no...

How to add esbuild to the rails asset pipeline

This are the steps I needed to do to add esbuild to an application that used the vanilla rails asset pipeline with sprockets before.

Preparations

  1. update Sprockets to version 4
  2. add a .nvmrc with your preferred node version (and install it)
  3. add gems jsbundling-rails and foreman to your Gemfile:
    gem 'jsbundling-rails'
    group :development, :test do
      gem 'foreman'
      # ...
    end
    
  4. bundle install
  5. run bin/rails javascript:install:esbuild in a console to prepare esbuild.
  6. run `yarn instal...

Sidekiq 7: Rate limiting with capsules

Sidekiq 7 adds a new feature called capsules.

Use cases:

  • a chrome queue limited to 1 for e.g. PDF processing to not overload the application server
  • an api queue, that limits a queue to 2 to protect the API server from too many requests in parallel

Example:

Sidekiq.configure_server do |config|
  # Edits the default capsule
  config.queues = %w[critical default low]
  config.concurrency = 5

  # Define a new capsule which ...

Threads and processes in a Capybara/Selenium session

TLDR: This card explains which threads and processes interact with each other when you run a Selenium test with Capybara. This will help you understand "impossible" behavior of your tests.


When you run a Rack::Test (non-Javascript) test with Capybara, there is a single process in play. It runs both your test script and the server responding to the user interactions scripted by your test.

A Selenium (Javascript) test has a lot more moving parts:

  1. One process runs your test script. This is the process you...

Git for Subversion users

This is for people recovering from Subversion.

Get an existing from the server for the first time

git clone git@example.com:repositoryname

See what's changed

git status

Check in locally

git commit -m "good description"

Push local commits to the server

git push

Get and merge updates from the server

git pull

Stage a file for the next local commit

git add file

Stage all files for the next local commit

git add .

Create a new local branch and check it out

git checkout -b branchname

...

Use find_in_batches or find_each to deal with many records efficiently

Occasionally you need to do something directly on the server -- like having all records recalculate something that cannot be done in a migration because it takes a long time.

Let's say you do something like this:

Project.all.each(&:recalculate_statistics!)

Even though you may have been successful with this on your development machine or the staging server, keep in mind that production machines often hold a lot more records. Using all may just work, even with lots of records, but when you iterate over such records and fetch a...

An incomplete guide to migrate a Rails application from paperclip to carrierwave

In this example we assume that not only the storage gem changes but also the file structure on disc.

A general approach

Part A: Create a commit which includes a script that allows you to copy the existing file to the new file structure.

Part B: Create a commit which removes all paperclip logic and replace it with the same code you used in the first commit

Part A

Here are some implementation details you might want to reuse:

  • Use the existing models to read the files from
  • Use your own carrierwave models to write t...

makandra cards: A knowledge base on web development, RoR, and DevOps

What is makandra cards?

We are makandra, a team of 60 web developers, DevOps and UI/UX experts from Augsburg, Germany. We have firmly anchored the sharing of knowledge and continuous learning in our company culture. Our makandra cards are our internal best practices and tips for our daily work. They are read worldwide by developers looking for help and tips on web development with Ruby on Rails and DevOps.

15 years ago – in 2009 – we wrote our first card. Since then, over 6000 cards have been created, not o...

How to fix broken font collisions in wkhtmltopdf

If you are using PDFKit / wkhtmltopdf, you might as well want to use custom fonts in your stylesheets. Usually this should not be a problem, but at times they include misleading Meta-information that leads to a strange error in the PDF.

The setup

  • The designer gave you two fonts named something like BrandonText-Regular and BrandonText-Bold. (With flawed Meta-information)
  • You have a HTML string to be rendered by PDFKit
  • For demonstration purposes, this strin...