How to monitor Sidekiq: A working example

In order to have monitoring for Sidekiq (like queue sizes, last run of Sidekiq) your application should have a monitoring route which returns a json looking like this:

{
  "sidekiq": {
    "totals": {
      "failed": 343938,
      "processed": 117649167
    },
    "recent_history": {
      "failed": {
        "2016-11-06": 1,
        "2016-11-07": 46,
        "2016-11-08": 0,
        "2016-11-09": 0,
        "2016-11-10": 0
      },
      "processed": {
        "2016-11-06": 230653,
        "2016-11-07": 230701,
        "2016-11-08"...

mceachen/closure_tree: Easily and efficiently make your ActiveRecord models support hierarchies

Closure_tree lets your ActiveRecord models act as nodes in a tree data structure.

This promises a few improvements over the battle-tested ancestry gem, such as:

  • Better performance
  • Pre-ordered trees (painful to do with ancestry)
  • Holds a mutex during tree manipulations (an issue with ancestry, where concurrent updates can cause deadlocks and corrupt data).

It has some more moving parts than ancestry though (see below).

Implementation
--------------...

Detecting if a Ruby gem is loaded

Detect if a gem has been activated

A gem is activated if it is either in the current bundle (Gemfile.lock), or if you have manually activated it using Kernel#gem (old-school).

To detect if e.g. activerecord has been activated:

if Gem.loaded_specs.has_key?('activerecord')
  # ActiveRecord was activated
end

Detect if a particular gem version has been activated

To detect if e.g. activerecord ma...

makandra/gemika: Helpers for testing Ruby gems

We have released a new library Gemika to help test a gem against multiple versions of Ruby, gem dependencies and database types.

Here's what Gemika can give your test's development setup (all features are opt-in):

  • Test one codebase against multiple sets of gem dependency sets (e.g. Rails 4.2, Rails 5.0).
  • Test one codebase against multiple Ruby versions (e.g. Ruby 2.1.8, Ruby 2.3.1).
  • Test one codebase against multiple database types (currently MySQL or PostgreSQL).
  • Compute a matrix of all possib...

Fix "libmysqlclient.so.20: cannot open shared object file: No such file or directory"

This error can be caused by the mysql2 gem under mysterious circumstances. You need to remove it with gem uninstall mysql2 and then reinstall it (or just run bundle).

gem pristine mysql2 will not be enough.

Bundler returns different error codes depending on what went wrong

When you are calling Bundler from your shell scripts, you might find it useful that a failed bundle call returns a different error code depending on the type of problem.

A list of error codes can be found here.

Minidusen: Low-tech record filtering with LIKE queries

We have a new gem Minidusen which extracts Dusen's query parsing and LIKE query functionality.

Minidusen can no longer index text in MySQL FULLTEXT columns, which was hardly used and didn't always help performance due to the cost of reindexing.

Minidusen is currently compatible with MySQL, PostgreSQL, Rails 3.2, Rails 4.2 and Rails 5.0.

Basic Usage

Our example will be a simple address book:

class Contact < ActiveRecord::Base
  validates_presence_of :name, :street, :city, :e...

Geordi 1.5.1 released

  • Improve geordi cucumber: Only attempt @solo run when the specified files contain the @solo tag, skip @solo run if any filename is passed with a line number (e.g. features/example.feature:3)
  • Improve geordi deploy: Find stages by their prefix (e.g. s -> staging, m -> makandra), bundle if needed, check the selected stage exists
  • Improve geordi server: Takes port as argument (e.g. geordi ser 3001), option --public (-P) starts the server with -b 0.0.0.0 to make it accessible from other machines in the local network, e.g. ...

Capistrano: exclude custom bundle groups for production deploy

Capistrano is by default configured to exclude the gems of the groups development and test when deploying to the stages production and staging. Whenever you create custom groups in your Gemfile, make sure to exclude these, if they should not be deployed to the servers. The gems of these groups might not be loaded by rails, however, the deployment process will take longer as the gems will be downloaded and installed to the server.

e.g. to exclude the groups cucumber and deploy, add the following to `config/deploy/production.rb...

There is no real performance difference between "def" and "define_method"

You can define methods using def or define_method. In the real world, there is no performance difference.

define_method is most often used in metaprogramming, like so:

define_method :"#{attribute_name}_for_realsies?" do
  do_things
end

Methods defined via define_method are usually believed to have worse performance than those defined via def.
Hence, developers sometimes prefer using class_eval to define methods using def, like this:

class_eval "def #{attribute_name}_for_realsies?; do_things; end"

You can be...

Download Ruby gems without installing

You can download .gem files using gem fetch:

gem fetch activesupport consul

This will produce files like active-support-5.0.0.gem and consul-0.12.1.gem in your working directory.

Dependencies will not be downloaded.

Ruby 2.3 brings Array#dig and Hash#dig

#dig lets you easily traverse nested hashes, arrays, or even a mix of them. It returns nil if any intermediate value is missing.

x = {
  foo: {
    bar: [ 'a', { baz: 'x' } ]
  }
}

x.dig(:foo, :bar) # => [ 'a', { baz: 'x' } ]
x.dig(:foo, :bar, 1, :baz) # => "x"
x.dig(:foo, :wronk, 1, :baz) # => nil

There is a tiny gem that backports this.

How to organize monkey patches in Ruby on Rails projects

As your Rails project grows, you will accumulate a number of small patches. These will usually fix a bug in a gem, or add a method to core classes.

Instead of putting many files into config/initializers, I recommend to group them by gem in lib/ext:

lib/
  ext/
    factory_girl/
      mixin.rb
    carrierwave/
      change_storage.rb
      fix_cache_ids.rb
      sanitize_filename_characters.rb
    ruby/
      range/
        covers_range.rb
      array/
        dump_to_excel.rb
        xss_aware_join.rb
      enumerable/
    ...

rroblak/seed_dump

This gem gives you a rake task db:seed:dump do create a db/seeds.rb from your current database state.

The generated db/seeds.rb will look this:

Product.create!([
  { category_id: 1, description: "Long Sleeve Shirt", name: "Long Sleeve Shirt" },
  { category_id: 3, description: "Plain White Tee Shirt", name: "Plain T-Shirt" }
])
User.create!([
  { password: "123456", username: "test_1" },
  { password: "234567", username: "test_2" }
])

Install MySQL 5.6 in Ubuntu 16.04

Instead of using this hack you might want to use MariaDB 10.x which can work with both old and new apps.


An alternative could be to use the MySQL Docker image which is still updated for 5.6.


Ubuntu 16.04 only provides packages for MySQL 5.7 which has a range of backwards compatibility issues with code written against older MySQL versions.

Oracle maintains a list of official APT repositories for MySQL 5.6, but those repositories do...

Using Bumbler to Reduce Runtime Dependencies - The Lean Software Boutique

Tool to show you which gems are slow to load:

➜  git:(master) ✗ bundle exec bumbler
[#################################################                             ]
(49/65) travis-lint...
Slow requires:
    110.21  render_anywhere
    147.33  nokogiri
    173.83  haml
    179.62  sass-rails
    205.04  delayed_job_active_record
    286.76  rails
    289.36  mail
    291.98  capistrano
    326.05  delayed_job
    414.27  pry
    852.13  salesforce_bulk_api

Stackprof - sampling call-stack profiler for ruby

Stackprof is a sampling call-stack profile for Ruby 2.1+.

Instead of tracking all method calls, it will simply collect the current stack trace of a running program at fixed intervals. Methods that appear on top of the stack trace most often, are the methods your program spends most of its time in.

The big advantage of this is that it is very fast. You can even enable it in production and collect real performance data. See the README on how to add it as a middleware. It will dump its data to the tmp directory.

Sampling is by default base...

djberg96/sys-filesystem: A Ruby library for getting filesystem information

Uses FFI and works all relevant operating systems.

If you'd try to do it yourself, you'd have to use FFI which is a bit awkward (see the gem's code), or call commands like df -B1 and search the strings for your relevant data.
That gem takes the pain away and works nicely.

Testing ActiveRecord validations with RSpec

Validations should be covered by a model's spec.

This card shows how to test an individual validation. This is preferrable to save an entire record and see whether it is invalid.

Recipe for testing any validation

In general any validation test for an attribute :attribute_being_tested looks like this:

  1. Make a model instance (named record below)
  2. Run validations by saying record.validate
  3. Check if record.errors[:attribute_being_tested] contains the expected validation error
  4. Put the attribute into a valid state
  5. Run...

Bundler: Install gems behind a proxy

To install gems Bundler needs to be able to talk to https://api.rubygems.org.

If you are behind a proxy you can use the https_proxy environment variable:

https_proxy=http://myproxy:123 bundle install

Note that if there is no https_proxy env variable, Bundler will also look for a http_proxy env variable.

With Capistrano

Ideally the server you're deploying on exports an https_proxy variable for all shells.

If you don't have control over the server setup, you can also add this to your Capistrano config:
...

NoMethodError: undefined method `cache' for Gem:Module

I got this error when running Rails 2.3 tests for Rails LTS. More stacktrace:

NoMethodError: undefined method `cache' for Gem:Module
    /vagrant/rails-2-3-lts-repository/railties/lib/rails_generator/lookup.rb:212:in `each'
    /vagrant/rails-2-3-lts-repository/railties/lib/rails_generator/lookup.rb:146:in `to_a'
    /vagrant/rails-2-3-lts-repository/railties/lib/rails_generator/lookup.rb:146:in `cache'
    /opt/vagrant_ruby/lib/ruby/1.8/fileutils.rb:243:in `inject'
    /vagrant/rails-2-3-lts-repository/railties/l...

Showing a custom maintenance page while deploying

Note

The maintenance mode is enabled on all application server as soon as the file /public/system/maintenance.html is present.

Installation

Add this line to your application's Gemfile:

 gem 'capistrano', '~> 3.0'
 gem 'capistrano-maintenance', '~> 1.0'

Add this line to you application's Capfile:

require 'capistrano/maintenance'

Enable task

Present a maintenance page to visitors. Disables your application's web interface by writing a #{maintenance_basename}.html file to each web server. The servers m...

Make Nokogiri use system libxml2

The nokogiri gem provides different packages for several platforms. Each platform-specific variant ships pre-built binaries of libxml2, e.g. x86_64-linux includes binaries for 64bit Linux on Intel/AMD. This significantly speeds up installation of the gem, as Nokogiri no longer needs to compile libxml2.

However, this also means that for each security issue with libxml2, Nokogiri maintainers have to update their pre-built binaries and release a new version of the gem. Then, you need to update and ...

Stop using bundle exec

Install this gem and stop using bundle exec or even Geordi's handy b. Yay!