Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)

General

A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch

This card deals with specifics concerning the use of searchkick.

Step 1: Make Opensearch available for Searchkick

In your Gemfile

# Search
gem 'searchkick'                   # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'

in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:

SEARCHKICK_CLIENT_T...

RSpec: Executing specs by example id (or "nesting index")

There are several ways to run a single spec. I usually copy the spec file path with the line number of the example and pass it to the RSpec binary: bin/rspec spec/models/user_spec.rb:30 (multiple line numbers work as well: :30:36:68). Another is to tag the example with focus: true or to run the example by matching its name.

In this card I'd like to ...

Headless Chrome: Changing the Accept-Language header is not possible

It seems like changing the HTTP_ACCEPT_LANGUAGE is not possible for a headless chrome.

  • On Ubuntu the headless Chrome derives the Accept-Language from the operation system
  • Adding the option options.add_argument('--lang=de') to the Capybara::Selenium::Driver has no effect
  • Adding the preference options.add_preference('intl.accept_languages', 'de') to the Capybara::Selenium::Driver has only effects if the --headless option is skipped (see bug ticket #775911)
  • Cha...

Do not use transparent PNGs for iOS favicons

Safari on iOS accepts an apple-touch-icon favicon that is used for stuff like desktop bookmarks. Always define a solid background color for them.

If you use PNGs with a transparent background, Safari will use just set a black background on your pretty icon. This is almost never what you want.
You can fix that by applying a white background via ImageMagick like this:

convert a...

Careful when using Time objects for generating ETags

You can use ETags to allow clients to use cached responses, if your application would send the same contents as before.

Besides what "actually" defines your response's contents, your application probably also considers "global" conditions, like which user is signed in:

class ApplicationController < ActionController::Base

  etag { current_user&.id }
  etag { current_user&.updated_at }

end

Under the hood, Rails generates an ETag header value like W/"f14ce3710a2a3187802cadc7e0c8ea99". In doing so, all objects from that etagge...

Spreewald, Cucumber: Selector for the nth Element

The recommended additional setup of the spreewald gem, a useful set of cucumber steps, includes adding a file for defining custom selectors which can be used as prose within steps:

When I follow "Edit" within the controls section

Where the controls section can be any arbitrary defined css selector within selectors.rb


Often it can be useful to select the nth element of a specific selector. Luckily, this can ...

Video transcoding: Web and native playback overview (April 2020)

Intro

Embedding videos on a website is very easy, add a <video> tag to your source code and it just works. Most of the time.

The thing is: Both the operating system and Browser of your client must support the container and codecs of your video. To ensure playback on every device, you have to transcode your videos to one or more versions of which they are supported by every device out there.

In this card, I'll explore the available audio and video standards we have right now. The goal is to built a pipeline that...

Git diff: Deemphasizing code that was only moved around

In long diffs, it can become impossible to spot small changes in larger blocks of moved code. This may be either a method that was moved from the top to the bottom of a file, or a long test file that was split in many.

Fortunately, Git offers a special highlighting mode that directs the reader's attention to relevant code parts:

git diff --color-moved=dimmed-zebra

It will dim lines that were moved around without changes, and highlight changed lines.
To easily use dimmed-zebra mode, configure an alias:

# ~/.gitconfig
[alias]
 ...

How to evaluate CSS media queries in JavaScript

To make CSS rules dependent on the screen size, we use media queries:

@media (max-width: 500px) {
  // rules for screen widths of 500px or smaller
}

Browsers will automatically enable and disable the conditional rules as the screen width changes.

To detect responsive breakpoints from JavaScript, you may use the global matchMedia() function. It is supported in all brow...

RSpec's hash_including matcher does not support nesting

You can not use the hash_including argument matcher with a nested hash:

describe 'user' do
  let(:user) { {id: 1, name: 'Foo', thread: {id: 1, title: 'Bar'} }

  it do 
    expect(user).to match(
      hash_including(
        id: 1, thread: {id: 1}
      )
    )
  end
end  

The example will fail and returns a not very helpful error message:

expected {:id => 1, :name => "Foo", :thread => {:id => 1, :title => "Bar"}} to...

RSpec: automatic creation of VCR cassettes

This RailsCast demonstrated a very convenient method to activate VCR for a spec by simply tagging it with :vcr.

For RSpec3 the code looks almost the same with a few minor changes. If you have the vcr and webmock gems installed, simply include:

# spec/support/vcr.rb
VCR.configure do |c|
  c.cassette_library_dir = Rails.root.join("spec", "vcr")
  c.hook_into :webmock
end

RSpec.configure do |c|
  c.around(:each, :vcr) do |example|
    name = example.metadata[:full_descripti...

Use CSS "text-overflow" to truncate long texts

When using Rails to truncate strings, you may end up with strings that are still too long for their container or are not as long as they could be. You can get a prettier result using stylesheets.

The CSS property text-overflow: ellipsis has been around for quite a long time now but since Firefox did not support it for ages, you did not use it. Since Firefox 7 you can!

Note that this only works for single-line texts. If you want to truncate tests across multiple lines, use a JavaScript solution like...

How to run a small web server (one-liner)

Sometimes you just want to have a small web server that serves files to test something.

Serve the current directory

On Ruby 1.9.2+ you can do the following ("." for current directory). You might need to gem install webrick on modern Rubies.

ruby -run -ehttpd . -p8000

Python 2.x offers a similar way.

python -m SimpleHTTPServer 8000 .

This is the same way with Python 3.x

python -m http.server

In both cases your web server is single-threaded and will block when large files are being downloaded from you.

WEBrick ...

Why preloading associations "randomly" uses joined tables or multiple queries

ActiveRecord gives you the :include option to load records and their associations in a fixed number of queries. This is called preloading or eager loading associations. By preloading associations you can prevent the n+1 query problem that slows down a many index view.

You might have noticed that using :include randomly seems to do one of the following:

  1. Execute one query per involved table with a condit...

What we know about PDFKit

What PDFKit is

  • PDFKit converts a web page to a PDF document. It uses a Webkit engine under the hood.
  • For you as a web developer this means you can keep using the technology you are familar with and don't need to learn LaTeX. All you need is a pretty print-stylesheet.

How to use it from your Rails application

  • You can have PDFKit render a website by simply calling PDFKit.new('http://google.com').to_file('google.pdf'). You can then send the...

Check that an element is hidden via CSS with Spreewald

If you have content inside a page that is hidden by CSS, the following will work with Selenium, but not when using the Rack::Test driver. The Selenium driver correctly only considers text that is actually visible to a user.

Then I should not see "foobear"

This is because the Rack::Test driver does not know if an element is visible, and only looks at the DOM.

Spreewald offers steps to check that an element is hidden by CSS:

Then "foo" should be hidden

You can also check that an el...

Rails: Example on how to extract domain independent code from the `app/models` folder to the `lib/` folder

This cards describes an example with a Github Client on how to keep your Rails application more maintainable by extracting domain independent code from the app/models folder to the lib/ folder. The approach is applicable to arbitrary scenarios and not limited to API clients.

Example

Let's say we have a Rails application that synchronizes its users with the Github API:

.
└── app
    └── models
        ├── user
        │   ├── github_client.rb
        │   └── sychronizer.rb
        └── user.rb

In this example the app folder ...

Sass: How to get rid of deprecation warnings in dependencies

sass >= 1.35.0 has the option quietDeps and silenceDeprecations to silence deprecation warnings from dependencies.

Below there are a few examples for different build tools how to set the Sass options.

Webpacker

const sassLoaderConfig = environment.loaders.get('sass')
const...

Automatic Log Rotation in Rails

Rails log files rotate automatically when they reach approx. 100MB:

$ ls -lh log/
-rw-r--r-- 1 user group  55M Sep 15 09:54 development.log
-rw-r--r-- 1 user group 101M Aug 22 13:45 development.log.0

This behavior is a built-in feature of Ruby's standard Logger class, which Rails uses by default.

To control the maximum file size, set config.log_file_size in yo...

ActiveRecord: Cleaning up your database with ignored_colums

Leaving old unused DB columns around after a migration is confusing for other developers. However, dropping columns too eagerly might also cause problems and extra work. If you want to mark columns for future deletion, but you are unsure, whether you can simply drop them right now, use these tools:

Add a comment to your DB schema

With schema comments you can add a comment like LEGACY as of yyyy-mm-dd to your DB schema.

Ignore the column

With [...

ActiveType::Object: Be careful when overriding the initialize method

Background:

ActiveType::Object inherits from ActiveRecod::Base and is designed to behave like an ActiveRecord Object, just without the database persistence.

Don't remove any of the default behavior of the initialize method!

If you have a class which inherits from ActiveType::Object and you need to override the #initialize method, then you should be really careful:

  • Always pass exactly one attribute. ActiveRecod::Base objects really want to get their arguments processable as keyword arguments. Don't change the syntax, or y...

Opt out of selenium manager's telemetry

If you use the selenium-webdriver gem, it will sneakily phone home once every hour whenever you run a browser based feature spec.

Check if you're affected

Check if ~/.cache/selenium/se-metadata.json exists. (It contains a "ttl" timestamp of its last/next anaytics call. You can parse it with Ruby's Time.at.)

Opt out

You can opt out either globally:

# .bashrc
export SE_AVOID_STATS=true

or project based

# spec_helper...

Rails: Rest API post-mortem analysis

This is a personal post-mortem analysis of a project that was mainly build to provide a REST API to mobile clients.

For the API backend we used the following components:

  • Active Model Serializer (AMS) to serializer our Active Record models to JSON.
  • JSON Schema to test the responses of our server.
  • SwaggerUI to document the API.

It worked

The concept worked really good. Here are two points that were extraordinary compared to normal Rails project with many UI components:

  • Having a Rails application, that has no UI components (only...

Simple database lock for MySQL

Note: For PostgreSQL you should use advisory locks. For MySQL we still recommend the solution in this card.


If you need to synchronize multiple rails processes, you need some shared resource that can be used as a mutex. One option is to simply use your existing (MySQL) database.

The attached code provides a database-based model level mutex for MySQL. You use it by simply calling

Lock.acquire('string to synchronize on') do
  # non-th...