PostgreSQL: How to show table sizes

When you have a large PG database, you may want to find out which tables are consuming the most disk space.
You can easily check this using the following SQL statement from the PostgreSQL wiki.

SELECT nspname || '.' || relname AS "relation",
    pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size"
  FROM pg_class C
  LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
  WHERE nspname NOT IN ('pg_catalog', 'information_schema')
    AND C.relkind <> 'i'
    AND nspname !~ '^pg_toast'
  ORDER BY pg_tot...

Fixing wall of net/protocol warnings

After upgrading to Rails 6.1.7.2 one of our apps printed a wall of warnings while booting:

/var/www/app/shared/bundle/ruby/2.6.0/gems/net-protocol-0.2.1/lib/net/protocol.rb:68: warning: already initialized constant Net::ProtocRetryError
/home/deploy-app/.rbenv/versions/2.6.10/lib/ruby/2.6.0/net/protocol.rb:66: warning: previous definition of ProtocRetryError was here
/var/www/app/shared/bundle/ruby/2.6.0/gems/net-protocol-0.2.1/lib/net/protocol.rb:214: warning: already initialized constant Net::BufferedIO::BUFSIZE
/home/deploy-app/.rben...

PSA: Chrome and Firefox do not always clear session cookies on exit

Cookies without an expiration timestamp are called "session cookies". [1] They should only be kept until the end of the browsing session.

However, when Chrome or Firefox are configured to reopen tabs from last time upon start, they will keep session cookies when closing the browser. This even applies to tabs that were closed before shutting down the browser.

This is by design in Chrome and [Firefox](https://bugzilla.mozilla.org/buglist.cgi?bug_id=337551,345830,358042,362212,36...

Solving "cannot remove Object::ClassMethods"

Most likely you run rake and your code is causing an exception which is not the one shown in your terminal.

Rails tries to catch this exception and clean up constants but -- while it's still booting up -- fails on this which causes another exception:

rake aborted!
cannot remove Object::ClassMethods

Running rake with the --trace parameter will give you no love; the backtrace is useless in most cases.

Try these approaches:

First: Check if there is a helpful error message

  • Ha...

Make "rake notes" learn about Haml, Sass, CoffeeScript, and other file types

Rails comes with a Rake task notes that shows code comments that start with "TODO", "FIXME", or "OPTIMIZE".

While it's generally not good practice to leave them in your code (your work is not done until it's done), in larger projects you will occasionally have to use them as other parts of the application that you depend upon are not yet available.
To keep track of them, run rake notes. Its output looks something like this:

$ rake notes
app/controllers/fron...

Make Less interpret the escape codes in a logfile

The unix command line tool less is a good choice for browsing logfiles. In the standard configuration, though, it does not interpret the escape sequences used in the rails logfiles. To enable this type:

less -R my_logfile.log

You can also have an alias to save yourself the typing

alias less='less -R'

8 steps for fixing other people's code

Guide how to make fixes in other people's GitHub repositories. It's basically "Open Source Development 101".

Way back in mid-2007, when Rails 1.2 was the new hotness and GitHub was still a year away from crawling out of the primordial internet soup, prolific open source contributor Dr Nic wrote an article titled “8 steps for fixing other people’s code”. (...)

Here in the fantastical future world of 2012, while we still don’t have hoverboards or household nuclear fusion, we do have some great tools that make fixing other people’s code...

Geordi 1.0 released

Geordi 1.0 features a command line application geordi, that holds most of Geordi's previous commands.

New features

  • command help and usage examples right within geordi (geordi help and geordi help <command>)

  • quick command access: type just the first few letters of a command, e.g. geordi rs or geordi dev[server]

  • command dependencies, e.g. geordi rspec invokes geordi bundle-install (which bundles only if needed)

  • no cluttered /usr/bin, but all commands in one handy tool

  • template for easily adding new...

Freeze (vendor, unpack) a single Ruby gem with and without Bundler

When you need to patch an existing gem, one way is to "vendor" the gem by copying it into the vendor/gems directory of your Rails project. You can then make any changes you require and Rails will use the vendored version of the gem after a server restart. Unfortunately you need to perform some additional steps to marry Rails and the copied gem. This notes describes what to do.

With Bundler

This is super-painful. If you just copy the gem to vendor/gems, Rails will complain:

Unpacked gem foolib in vendor/gems has no s...

Always store your Paperclip attachments in a separate folder per environment

tl;dr: Always have your attachment path start with :rails_root/storage/#{Rails.env}#{ENV['RAILS_TEST_NUMBER']}/.


The directory where you save your Paperclip attachments should not look like this:

storage/photos/1/...
storage/photos/2/...
storage/photos/3/...
storage/attachments/1/...
storage/attachments/2/...

The problem with this is that multiple environments (at least development and test) will share the same directory structure. This will cause you pain eventually. Files will get overwritten and...

RSpec: How to test the content of a flash message in a request spec

The ActionDispatch module of Rails gives you the helper method flash to access the flash messages in a response.

describe PostsController, type: :request do

  describe 'update' do

    it 'shows a success message on update' do
      post_record = create(:post)

      put "/posts/#{post_record.id}"

      # Same as @request.flash[:alert]
      expect(flash[:alert]).to eq('Post updated successfully.')
    end

  end


end

Headless Chrome: Changing the Accept-Language header is not possible

It seems like changing the HTTP_ACCEPT_LANGUAGE is not possible for a headless chrome.

  • On Ubuntu the headless Chrome derives the Accept-Language from the operation system
  • Adding the option options.add_argument('--lang=de') to the Capybara::Selenium::Driver has no effect
  • Adding the preference options.add_preference('intl.accept_languages', 'de') to the Capybara::Selenium::Driver has only effects if the --headless option is skipped (see bug ticket #775911)
  • Cha...

makandra/gemika: Helpers for testing Ruby gems

We have released a new library Gemika to help test a gem against multiple versions of Ruby, gem dependencies and database types.

Here's what Gemika can give your test's development setup (all features are opt-in):

  • Test one codebase against multiple sets of gem dependency sets (e.g. Rails 4.2, Rails 5.0).
  • Test one codebase against multiple Ruby versions (e.g. Ruby 2.1.8, Ruby 2.3.1).
  • Test one codebase against multiple database types (currently MySQL or PostgreSQL).
  • Compute a matrix of all possib...

We have deprecated Rack::SteadyETag

Rack::SteadyETag was a Rack middleware that generates the same default ETag for responses that only differ in XOR-masked CSRF tokens or CSP nonces.

We have deprecated Rack::SteadyETag. We instead recommend reconfiguring your Rails app so two requests to the same resource produce the same HTML for a given user.

ActionMailer: How to send a test mail directly from the console

If your rails application is unable to send mails, it might be useful to debug your settings using the rails console. Here is a snippet that shows the current settings and lets you send a test mail directly from the console:

mailer = ActionMailer::Base.new

# check settings:
mailer.delivery_method # -> :smtp
mailer.smtp_settings # -> { address: "localhost", port: 25, domain: "localhost.localdomain", user_name: nil, password: nil, authentication: nil, enable_starttls_auto: true }

# send mail:
mailer.mail(from: 'sender@example.com', ...

Heads up: Byebug has problems with zeitwerk

I encountered a unlucky behavior of byebug 11.1.3 (the most recent version at time of writing) when using it with Rails 6 and it's new autoloading component, zeitwerk. There already is a issue for that, so I hope it will be fixed with a future release.

The following test succeeds:

  context 'factories' do
    let(:test_case) { FactoryBot.create(:test_case) }
    it 'are valid' do
      expect(test_case).to be_valid
    end
  end

But when I did the same in byebug the foll...

Sprites with Compass

Using CSS sprites for background images is a technique for optimizing page load time by combining smaller images into a larger image sprite.

There are ongoing arguments on how useful this still is, as modern browsers become more comfortable to load images in parallel. However, many major websites still use them, for example amazon, [facebook](...

Fun with Ruby: Returning in blocks "overwrites" outside return values

In a nutshell: return statements inside blocks cause a method's return value to change. This is by design (and probably not even new to you, see below) -- but can be a problem, for example for the capture method of Rails.


Consider these methods:

def stuff
  puts 'yielding...'
  yield
  puts 'yielded.'
  true
end

We can call our stuff method with a block to yield. It works like t...

RSpec: How to compare ISO 8601 time strings with milliseconds

Rails includes milliseconds in Time / DateTime objects when rendering them as JSON:

JSON.parse(User.last.to_json)['created_at']
#=> "2001-01-01T00:00:00.000+00:00"

In RSpec you might want to use .to_json instead of .iso8601 to use the build-in eq matcher:

it 'returns the created at attribute of a user' do
  get '/users/1'
  
  expect(JSON.parse(response.body)['created_at']).to eq(Time.parse('2001-01-01').to_json)
end

Otherwise the strings do not match:

DateTime.parse('2001-01-01').to_s (will defa...

Speed up JSON generation with oj

Using this gem I could get JSON generation from a large, nested Ruby hash down from 200ms to 2ms.

Its behavior differs from the default JSON.dump or to_json behavior in that it serializes Ruby symbols as ":symbol", and that it doesn't like an ActiveSupport::HasWithIndifferentAccess.

There are also some issues if you are on Rails < 4.1 and want it to replace #to_json (but you can always just call Oj.dump explicitely).

Security warning: Oj does not escape HTML entities in JSON
---------...

Postgresql: Paginate and count in one query using window functions

When paginating records, we usually need to know the number of total records in order to render pagination links. Popular pagination libraries like will_paginate or Kaminari do this for us by simply issuing an extra query, like this:

SELECT post.* FROM posts LIMIT 20 OFFSET 100;

SELECT COUNT(*) FROM posts;   

This is fine most of the time. But rarely, you might have very complicated WHERE conditions or a subquery that takes time to run. In thes...

Dynamic conditions for belongs_to, has_many and has_one associations

Note: Consider not doing this. Use form models or vanilla methods instead.


The :conditions option for Rails associations cannot take a lambda. This makes it hard to define conditions that must be evaluated at runtime, e.g. if the condition refers to the current date or other attributes.

A hack to fix this is to use faux string interpolation in a single-quoted :conditions string:

class User < ActiveRecord::Base
  has_many :contracts
  has_one :current_contract, :class_name => 'Contract', :conditions => '...

Perform Sidekiq jobs immediately in development

# config/initializers/sidekiq.rb

# Perform Sidekiq jobs immediately in development,
# so you don't have to run a separate process.
# You'll also benefit from code reloading.
if Rails.env.development?
  require 'sidekiq/testing'
  Sidekiq::Testing.inline!
end

How to recognize CVE-2019-5418

If you get requests with values for formats like this:

{:locale=>[:de], :formats=>["../../../../../../../../../../etc/services{{"], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :haml]}

or fails like this:

Invalid query parameters: invalid %-encoding (../../../../../../../../../etc/passwd%%0000.html)

Someone tries to exploit CVE-2019-5418.
If you use the latest Rails (or latest Rails LTS) you're...