Linux Performance Analysis in 60,000 Milliseconds

You login to a Linux server with a performance issue: what do you check in the first minute?

uptime
dmesg | tail
vmstat 1
mpstat -P ALL 1
pidstat 1
iostat -xz 1
free -m
sar -n DEV 1
sar -n TCP,ETCP 1
top

Also see:

Upgrading from Capistrano 2 to 3

Capistrano 3 is a major rework of the framework and requires several adjustments to your deploy configuration files. The biggest change is that they moved away from their custom DSL and use Rake instead. For connecting with and operating on the servers, they bring a new gem SSHKit which does the heavy lifting. It's SSHKit's DSL that is used anywhere inside the Rake tasks. See #Resources at the bottom for examples.

Step 1: Upgrade guide

For migration from 2 to 3, follow this tutorial: [Capistrano 3 Upgrade Guide](https://semaphorec...

Fix error: Missing the mysql2 gem

So you got this error, even though your Gemfile bundles mysql2:

!!! Missing the mysql2 gem. Add it to your Gemfile: gem 'mysql2'

or

Please install the mysql adapter: `gem install activerecord-mysql-adapter` (mysql is not part of the bundle. Add it to Gemfile.)

The reason for this confusing error message is probably that your Gemfile says mysql2, but your database.yml still uses the mysql adapter. Change it to use the mysql2 adapter:

development:
  adapter: mysql2
  database: myproject_developm...

How to mount a legacy database to migrate data

There are many approaches out there how you can import data from a legacy application to a new application. Here is an approach which opens two database connections and uses active record for the legacy system, too:

1. Add you database information to you config/database.yml.

data_migration:
  database: your_application_data_migration

2. Create a separate application record for the data migration, e.g. in app/data_migration/migration_record.rb. You will need to create an app/data_migration.rb class first.

class DataMig...

Designing HTML emails

The 90s are calling: they want their tables back. Unfortunately, you need them all for laying out your HTML emails.

Email client HTML rendering is way more scattered than browser HTML. While you might have a pretty good understanding of what features and patterns you can use to support all major browsers, I doubt anyone masters this craft for HTML email clients.

The only way to ensure your email looks good (acceptable, at least) in all mail clients, is to check it. Litmus is your go-to solution for this (see below). W...

Bash: How to use colors in your tail output

Sometimes it's nice to have some coloring in your logs for better readability. You can output your logs via tail and pipe this through sed to add ANSI color annotations (which your console then interprets).

To print a log (e.g. rails log) and color all lines containing "FATAL" in red and all lines with "INFO" in green:

tail -f /path/to/log | sed --unbuffered -e 's/\(.*INFO.*\)/\o033[32m\1\o033[39m/' -e 's/\(.*FATAL.*\)/\o033[31m\1\o033[39m/'

Here are the ...

Run a script on the server

You have to specify the environment with -e env_name or RAILS_ENV=env_name if you want to run a script on the server.

at Rails 2 it's script/runner

bundle exec script/runner -e env_name path/to/script.rb argument1 argument2 ...

at Rails 3 it's rails runner

RAILS_ENV=env_name bundle exec rails runner path/to/script.rb argument1 argument2 ...

Retrieve the SQL query a scope would produce in ActiveRecord

Rails 3

User.active.to_sql

Rails 2

Use either the Edge Rider or fake_arel gem to get #to_sql backported to Rails 2.

If you don't want to use a gem for this, you can do this with vanilla Rails 2:

User.active.construct_finder_sql({})

How to fix gsub on SafeBuffer objects

If you have an html_safe string, you won't be able to call gsub with a block and match reference variables like $1. They will be nil inside the block where you define replacements (as you already know).

This issue applies to both Rails 2 (with rails_xss) as well as Rails 3 applications.

Here is a fix to SafeBuffer#gsub. Note that it will only fix the $1 behavior, not give you a safe string in the end (see below).

Example

def test(input)...

Geordi 1.3 released

Changes:

  • Geordi is now (partially) tested with Cucumber. Yay!
  • geordi cucumber supports a new @solo tag. Scenarios tagged with @solo will be excluded from parallel runs, and run sequentially in a second run
  • Support for Capistrano 2 AND 3 (will deploy without :migrations on Capistrano 3)
  • Now requires a .firefox-version file to set up a test firefox. By default now uses the system Firefox/a test Chrome/whatever and doesn't print warnings any more.
  • geordi deploy --no-migrations (aliased -M): Deploy with `cap ...

Installing multiple MySQL versions on the same Linux with mysql-sandbox

Ubuntu has a package mysql-sandbox that lets you install multiple MySQL versions into your user home:

  1. Install mysql-sandbox
sudo apt install mysql-sandbox
  1. Download the version of MySQL you want to use from mysql.com:
    https://dev.mysql.com/downloads/file/?id=480427
    Make sure to choose "Generic Linux" instead of "Ubuntu" so you get a .tar.gz instead of .deb

  2. cd into the directory the mysql binaries will be extracted to

mkdir -p ~/bin/sandbox_dist
cd ~/bin/sandbox_dist
  1. Build the sandbox
make_sandbo...

Copy to clipboard without flash (clipboard.js)

We used zeroclipboard.js in some of our projects but now we switched to clipboard.js because it does not rely on flash. Flash support of the major browsers has ended.

Some more advantages of clipboard.js:

  • it consists only of a single javascript file, so it does not trigger additional requests with rails
  • it automagically provides user feedback by selecting the text it has copied
  • it provides callbacks for success and error which make it easier to add custom behaviour after copying to the clipboar...

When does Webpacker compile?

Webpack builds can take a long time, so we only want to compile when needed.

This card shows what will cause Webpacker (the Rails/Webpack integration) to compile your assets.

When you run a dev server

While development it is recommended to boot a webpack dev server using bin/webpack-dev-server.

The dev server compiles once when booted. When you access your page on localhost before the initial compilation, the page may load without assets.

The ...

Unpoly 2: Don't try to download files through AJAX requests

Rails has the handy controller method send_file which lets us download files easily. We can decide whether the file should be downloaded (disposition: 'attachment') or shown in the browser (disposition: 'inline'). The default is disposition: 'attachment'.

Downloading files will not work when you are calling the controller action from an AJAX request. The browser will try to render the file and insert it in the DOM, which is never what you want.

Unpoly 2

Unpoly (sin...

AngularJS directive to format a text with paragraphs and new lines

If you are using Angular and want something like Rails' simple_format which HTML-formats a plain-text input into paragraphs and line breaks, this directive is for you.

Any HTML fragments inside that text will still be escaped properly.

Use it like this, where your text attribute specifies something available in your current scope:

<simple-format text="email.message"></simple-format>

This is the directive, in CoffeeScript syntax:

@app.directive 'simpleFor...

Render Sass stylesheets dynamically

If - for whatever reason - you have to render stylesheets dynamically, the following snippet might be of help. It emulates what "sprockets" would to when precompiling your assets, and give your stylesheets access to all the regular bells and whistles (like asset_path, proper @imports etc):

class DynamicStylesheetsController < ApplicationController

    def show
      logical_path = RELATIVE_PATH_TO_YOUR_TEMPLATE
      path = File.join(Rails.root, logical_path)
      template = Sass::Rails::SassTemplate.new(path)
      environment = ...

Test redirects to an external URL with Cucumber/Capybara

When a controller action redirects to an external URL (like http://somehost.com/some/path) you will find that this is hard to test with Cucumber and Capybara:

  • A non-Javascript Rack::Test scenario will just ignore the host and try to open /some/path in your local application
  • A Selenium test will actually follow the redirect, which you probably don't want either

There are two workarounds for this. You can use either, or a combination of both.

  1. Write a controller spec

Controller specs can test if a resp...

Asset Pipeline Basics

The Rails asset pipeline improves delivery of application assets (javascripts, stylesheets, images, fonts). Here are some basic facts about its inner workings.

No magic

Manifests are the handle on your assets:

app/assets/stylesheets/application.css # use via: stylesheet_link_tag 'application'

The asset pipeline only considers files you explicitly require within your manifest files. The most common directives used in manifests are require some/file and require_tree some/directory. Paths may be **relative to the current director...

Fix: esbuild assets are missing after capistrano deploy

Issue: You have an app using jsbundling-rails and esbuild. After deploy, the assets built by esbuild are missing in public/assets.

Solution: Add app/builds to your git repo (by adding a app/builds/.keep file).

Something in sprockets is caching paths and refuses to accept files in "unknown" locations.

How to reload a belongs_to association

To reload a single-item association in Rails 5+, call #reload_<association>:

post.reload_author

In older Railses you can say

post.author(true)

Careful when writing to has_many :through associations

tl;dr: Using has_many associations with a :through option can lead to lost or duplicate records. You should avoid them, or only use them to read records.

Consider this:

class User < ActiveRecord::Base
end

class Party < ActiveRecord::Base
  has_many :invitations
  has_many :users, through: :invitations, include: :user, order: 'users.name'
end

class Invitation < ActiveRecord::Base
  belongs_to :party
  belongs_to :user
  
  after_create :send_invite
  
  def send_invite
  ...

How to fix: Gems are unavailable although they are installed

  • If Rails or Rake are complaining about a missing gem that is listed in your Gemfile.lock and the listed version is properly installed, something is seriously wrong and needs to be fixed.
  • If you accidently executed bundle install some_gem although you wanted bundle update some_gem

What is wrong

Let's say your Gemfile asks for some-gem which you can see when running gem list but bundle show some-gem just gives you an error:

Could not find gem 'some-gem', in any of the sources

Another indicator: Doing a `...

Nicer alternatives to def_delegator or def_delegators

Delegating methods to other objects is often helpful but the syntax of both def_delegators and def_delegator is a complete mess that makes your code hard to read.

Consider these classes:

class Topic < ActiveRecord::Base
  def title
    "A title"
  end
  
  def category
    "Topic category"
  end
end

class Post < ActiveRecord::Base
  belongs_to :topic
  def_delegato...

Understanding database cleaning strategies in tests

TLDR: In tests you need to clean out the database before each example. Use :transaction where possible. Use :deletion for Selenium features or when you have a lot of MyISAM tables.

Understanding database cleaning

You want to clean out your test database after each test, so the next test can start from a blank database. To do so you have three options:

  • Wrap each test in a transaction which is rolled back when you're done (through DatabaseCleaner.strategy = :transaction or `config.use_transactional_fi...