Transfer records to restore database entries (with Marshal)

If you ever need to restore exact records from one database to another, Marshal might come in handy.

Marshal.dump is part of the ruby core and available in all ruby versions without the need to install anything. This serializes complete ruby objects including id, object_id and all internal state.

Marshal.load deserializes a string to an object. A deserialized object cannot be saved to database directly as the the dumped object was not marked dirty, thus rails does not see the need to save it, even if the object is not present in...

Ruby: Using `sprintf` to replace a string at fixed named references

The sprintf method has a reference by name format option:

sprintf("%<foo>d : %<bar>f", { :foo => 1, :bar => 2 }) # => 1 : 2.000000
sprintf("%{foo}f", { :foo => 1 })                      # => "1f"

The format identifier %<id> stands for different data types to be formatted, such as %f for floats:

sprintf('%f', 1) # => 1.000000

Example:

This is quite useful to replace ...

How to: Upgrade CarrierWave to 3.x

While upgrading CarrierWave from version 0.11.x to 3.x, we encountered some very nasty fails. Below are the basic changes you need to perform and some behavior you may eventually run into when upgrading your application. This aims to save you some time understanding what happens under the hood to possibly discover problems faster as digging deeply into CarrierWave code is very fun...

Whitelists and blacklists

The following focuses on extension allowlisting, but it is the exact same thing for content type allowlisting with the `content_ty...

PSA: Be super careful with complex `eager_load` or `includes` queries

TLDR

Using .includes or .eager_load with 1-n associations is dangerous. Always use .preload instead.

Consider the following ActiveRecord query:

BlogPost.eager_load(
  :comments
  :attachments,
).to_a

(Let's assume we only have a couple of blog posts; if you use pagination the queries will be more complicated, but the point still stands.

Looks harmless enough? It is not.

The problem

ActiveRecord will rewrite this into a query using LEFT JOINs which looks something like this:

SELECT "blog_posts...

Heads up: Quering array columns only matches equally sorted arrays

Given you have an array column like this:

create_table "users", force: :cascade do |t|
  t.integer "movie_ids", default: [], array: true
end

You might think that the following queries yield the same result:

User.where(movie_ids: [16, 17])
User.where(movie_ids: [17, 16])

Turn's out - they are not! They do care about array ordering more than I do.

To query for identical arrays independent of their order you have to either:

  1. Sort both the query and database content. If you're on Rails 7.1 you can use the new [`normal...

Spreewald, Cucumber: Selector for the nth element

The recommended additional setup of the spreewald gem, a useful set of cucumber steps, includes adding a file for defining custom selectors which can be used as prose within steps:

When I follow "Edit" within the controls section

Where the controls section can be any arbitrary defined css selector within selectors.rb


Often it can be useful to select the nth element of a specific selector. Luckily, this can ...

How to make sure that manual deploy tasks (scheduled in Pivotal Tracker) are executed on deploy (with Capistrano)

We regularly have tasks that need to be performed around a deploy. Be it to notify operations about changed application behavior, be it to run a little oneline script after the deploy. Most database-related stuff can be handled by migrations, but every once in a while, we have tasks that are much easier to be performed manually.

Writing deploy tasks

Here is how we manage the deploy tasks themselves:

  • Deploy tasks are written inside the Pivotal Tracker story description, clearly marked (e.g. with a headline "Deploy task")
  • We disting...

How to work around selenium chrome missing clicks to elements which are just barely visible

Chromedriver (or selenium-webdriver?) will not reliably scroll elements into view before clicking them, and actually not click the element because of that.

We've seen this happen for elements which are just barely in the viewport (e.g. the upper 2px of a 40px button). Our assumption is that the element is considered visible (i.e. Capybara::Selenium::ChromeNode#visible? returns true for such elements) but the Selenium driver wants to actually click the center of the element which is outside of the viewport.

We don't know who exactly i...

Don't use log level :debug in your production environments

Catch phrase

You don't want sensitive user data in your logs.

Background

Rails per default filters sensitive data like passwords and tokens and writes [FILTERED] to the logs. The code which is responsible for enabling that usually lives in filter_parameter_logging.rb (Rails.application.config.filter_parameters). Here is an example of a filtered log entry:

Unfiltered:
`User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."token" = $1 LIMIT $2 [["token", "secret-token"], ["LIMIT", 1]]`

After the filter is appl...

Upgrade Rails: Awareness list

Disclaimer

This card is a collection of guides and things to have in mind when upgrading to a specific version. It is not meant to be complete, so please feel free to contribute!

General workflows

Upgrade to Rails 7

Split your parallel tests by execution time and keep execution logs up to date

Both knapsack and parallel_tests have the option to split groups by historic execution time. The required logs for this might be outdated since you manually have to update and push them into your repository.

The following card includes an option how you can keep them consistently up to date with no extra effort locally and/or remotely.

How to always split by execution logs

Parallel Tests

The parallel_tests gem has the option flag `--group...

Git restore vs. reset for reverting previous revisions

The git doc states on the difference of these two commands:

  • git-restore[1] is about restoring files in the working tree from either the index or another commit. This command does not update your branch. The command can also be used to restore files in the index from another commit.
  • git-reset[1] is about updating your branch, moving the tip in order to add or remove commits from the branch. This operation changes the commit history.

git reset can also be used to restore th...

redirect_to and redirect

There are multiple ways to redirect URLs to a different URL in Rails, and they differ in small but important nuances.

Imagine you want to redirect the following url https://www.example.com/old_location?foo=bar to https://www.example.com/new_location?foo=bar.

Variant A

You can use ActionController::Redirecting#redirect_to in a controller action

class SomeController < ActionController::Base
  def old_location
    redirect_to(new_location_url(params.permit(:foo))) 
  end
end

This will:

  • It will redirect with a 302 st...

Do not use "permit!" for params

Rails' Strong Parameters enable you to allow only specific values from request params to e.g. avoid mass assignment.

Usually, you say something like params.permit(:email, :password) and any extra parameters would be ignored, e.g. when calling to_h.
This is excellent and you should definitely use it.

What is permit! and why is it dangerous?

However, there is also params.permit! whic...

Capybara: How to find the focused element

Capybara allows you to filter elements that are focused.

page.find(:fillable_field, focused: true) # Filtering only fillable inputs for performance reasons
page.find(:xpath, '//*', focused: true) # Filter all fields

Legacy approach

In older version, it was possible to use the :focus pseudo-class. This seems not to work in newer versions anymore.

find(':focus')

Postgres in Alpine docker container: sorting order might differ

In CI test runs I noticed that string sorting order changed after switching from a debian-based PostgreSQL docker image to one that is based on Alpine Linux.

Debian image sorting: bar Bar foo Foo
Alpine image sorting: Bar Foo bar foo

Explanation

Alpine Linux is a very slim linux distribution that results in small docker image sizes (roughly 100MB instead of 150MB), so it's a popular choice. However, it does not have all comman locales installed and does not use all locales that a user installs by default.
Postgres orders string co...

Use -webkit-line-clamp to natively truncate long (multi-line) texts with an ellipsis

Note: You won't need this for single lines of text. In this case it is better to just use the text-overflow property: Use CSS "text-overflow" to truncate long texts

You can use -webkit-line-clamp in your CSS/SASS to natively render an ellipsis (...) after a specific amount of lines for a multi-line text in your HTML.
Earlier, it was necessary to implement JavaScript solutions like Superclamp.js to enable this because the browser support has been rather limited...

CarrierWave: Processing images with libvips

When you write your next CarrierWave uploader, consider processing your images with libvips instead of ImageMagick.

Reasons for libvips

There are several upsides to using libvips over ImageMagick:

RubyMine and Rubocop: Performing safe autocorrects on save

  • Ctrl + Alt + S > search "rubocop on save"
  • Under "Inspections", check the highlighted box on rubocop -a

Caveat: This adds a little time overhead to saving. When you're editing many files at once (e.g. using "Replace All"), this may be inacceptable.

Balance your texts today with text-wrap: balance

So you have a heading that is just barely wider than the container it should fit into, and it wraps a single word to a new line and it's not really pretty?
Cry no more, for you can use text-wrap: balance today to fix that. At least in some browsers.

When browsers encounter a text-wrapping element with text-wrap: balance style, they will try breaking to a new line sooner, if it balances out the width of lines.

text-wrap: unset text-wrap: balance
Image ![Image](/makandr...

Git: Splitting up changes into several commits

Splitting up commits makes the process of reviewing often easier, since you can create several merge requests or review every commit one by one.

So when you find out that you have portions of the code that you initially didn't intend to change or when you do some refactoring along the current changes, you can use one of the following processes to split up the changes into several commits in a logical order:

#1 Splitting up the last n commits into m commits
#2 Adding changes to a previous commit
2.1 While adding new changes
2.2 S...

Best practice: How to manage versions in a package.json

It most cases it's not necessary to add a version constraint next to your packages in the package.json. Since all versions are saved in a lockfile, everyone running yarn install will get exactly the same versions. Yarn saves this lockfile as yarn.lock and npm as package-lock.json.

There are some exceptions, where you can consider adding a version constraint to the package.json:

  • You are not checking the lockfile into version control (not recommended)
  • A specific package has a bug in a more recent version
  • You want to ensure no...

Capo: putting <head> content into the right order

How you order elements in the <head> can have an effect on the (perceived) performance of the page.
This script helps you identify which elements are out of order.

Also available as a Chrome Extension. It shows the actual order of head elements, and suggests an optimal order for page performance.

How to pretty print all values in a Redis database

With this Ruby script you can print all values in a Redis database to your console (derived from this bash script).

Note: Do not run this command in production. This is for debugging purposes only.

def pretty_print_redis(redis)
  redis.keys.each_with_object({}) do |key, hash|
    type = redis.type(key)

    hash[key] = case type
    when 'string'
      redis.get(key)
    when 'hash'
      redis.hgetall(key)
    when 'list'
      redis.lrange(key, 0, -1)
    when 'set'
      redis.smembers(...