How to speed up JSON rendering with Rails

I was recently asked to optimize the response time of a notoriously slow JSON API endpoint that was backed by a Rails application.
While every existing app will have different performance bottlenecks and optimizing them is a rabbit hole of arbitrary depth, I'd like to demonstrate a few techniques which could help reaching actual improvements.

The baseline

The data flow examined in this card are based on an example barebone rails app, which can be used to reproduce the r...

How to query GraphQL APIs with Ruby

While most Rails Apps are tied to at least one external REST API, machine-to-machine communication via GraphQL is less commonly seen. In this card, I'd like to give a quick intro on how to query a given GraphQL API - without adding any additional library to your existing app.

Core aspects of GraphQL

Interacting with GraphQL feels a bit like querying a local database. You are submitting queries to fetch data in a given structure (like SELECT in SQL) or mutations to alter the database (similar to POST/PUT/DELETE in REST). You can ...

Rails npm packages will use an uncommon versioning scheme

When Rails releases a new version of their gems, they also release a number of npm packages
like @rails/activestorage or @rails/actioncable.

Unfortunately Rails uses up to 4 digits for their gem version, while npm only allows 3 digits and a pre-release suffix.

To map gem versions and npm versions, Rails is going to use a naming scheme like this:

Gem version npm version
7.0.0 7.0.0
7.0.1 7.0.100
...

Ruby: Retrieving and processing files via Selenium and JavaScript

This card shows an uncommon way to retrieve a file using selenium where JavaScript is used to return a binary data array to Ruby code.

The following code example retrieves a PDF but the approach also works for other file types.

require "selenium-webdriver"

selenium_driver = Selenium::WebDriver.for :chrome
selenium_driver.navigate.to('https://example.com')
link_to_pdf = 'https://blobs.example.com/random-pdf'

binary_data_array = selenium_driver.execute_script(<<-JS, link_to_pdf)
  const response = await fetch(arguments[0])

  if (!r...

Bundler: Enforce consistent platform versions

Some rubygems come in platform-specific versions (i.e. "x86_64-linux") in addition to the usual "ruby" platform. This is often is done for gems with native extensions that want to deliver precompiled binaries for ease of installation. However, there is an issue on some versions of Bundler (at least 2.3.x) that makes it inconsistent which versions of a gem will be installed.

Why this happens

On (very) old versions of Bundler, the Gemfile.lock could never indicate which version of a gem was installed, i.e. the Gemfile read `nokogiri (1....

Upgrading RubyGems to the last compatible version

Running gem update --system will install the latest version of RubyGems. However the latest version might not be compatible with your current version of Ruby.

To install the latest version that is compatible with your Ruby:

  • Go to the version history and find the latest compatible version (any Ruby constraint can be found in the right sidebar)
  • Append that version to the update command. E.g. to install the latest version compatible with Ruby 2:
    gem update --system 3.4.22
    

...

Ruby: How to make your ruby library configurable

You might know a few examples, where you configure some library via a block. One example is the Rails configuration:

Rails.application.configure do |config|
  config.enable_reloading = false
end

This card describes a simple example on how to make your ruby library configurable.

Example

module FooClient
  class Client
    class_attribute :config

    def self.configure
      self.config ||= Configuration.new
      yield(config)
    end

    def test
      uri = URI.parse(FooClient::Client.config.endpoint)
      Net:...

Rails: Example on how to extract domain independent code from the `app/models` folder to the `lib/` folder

This cards describes an example with a Github Client on how to keep your Rails application more maintainable by extracting domain independent code from the app/models folder to the lib/ folder. The approach is applicable to arbitrary scenarios and not limited to API clients.

Example

Let's say we have a Rails application that synchronizes its users with the Github API:

.
└── app
    └── models
        ├── user
        │   ├── github_client.rb
        │   └── sychronizer.rb
        └── user.rb

In this example the app folder ...

Ruby: Different ways of assigning multiple attributes

This card is a short summary on different ways of assigning multiple attributes to an instance of a class.

Using positional parameters

Using parameters is the default when assigning attributes. It works good for a small number of attributes, but becomes more difficult to read when using multiple attributes.

Example:

class User
  def initialize(salutation, first_name, last_name, street_and_number, zip_code, city, phone_number, email, newsletter)
    @salutation = salutation
    @first_name = first_name
    @last_name = last_nam...

routing-filter is broken with Rails 7.1

If you are using the routing-filter gem in your Rails 7.1 app for managing URL segments for locales or suffixes, you will notice that the filters do no longer apply, routes are broken and the necessary parameters are no longer extracted. That is because routing-filter patches Rails' find_routes-method to get the current path and apply its defined filters on it. These filters then modify the params that are handed over to your controller action. This way you receive a locale parameter from a ...

makandra cards: A knowledge base on web development, RoR, and DevOps

What is makandra cards?

We are makandra, a team of 60 web developers, DevOps and UI/UX experts from Augsburg, Germany. We have firmly anchored the sharing of knowledge and continuous learning in our company culture. Our makandra cards are our internal best practices and tips for our daily work. They are read worldwide by developers looking for help and tips on web development with Ruby on Rails and DevOps.

15 years ago – in 2009 – we wrote our first card. Since then, over 6000 cards have been created, not o...

Ruby: Following redirects with the http gem ("httprb")

When making requests using the http gem you might want to automatically follow redirects to get the desired response. This is supported but not the default behavior.

You can do so by using the .follow method.
This follows redirects for the following status codes: 300, 301, 302, 303, 307, 308

response = HTTP.get('https://www.example.com/redirect')
response.status # => 302
response.uri.to_s # => "https://www.example.com/redirect"

response = HTTP.follow.get('https://www.example.com/redirect')
response.status # => 200
response....

Using rack-mini-profiler (with Unpoly)

Debugging performance issues in your Rails app can be a tough challenge.

To get more detailed insights consider using the rack-mini-profiler gem.

Setup with Unpoly

Add the following gems:

group :development do
  gem 'memory_profiler'
  gem 'rack-mini-profiler'
  gem 'stackprof'
end

Unpoly will interfere with the rack-mini-profiler widget, but configuring the following works okayish:

// rack-mini-profiler + unpoly
if (process...

Ruby: How to connect to a host with expired SSL certificate

If you need to make an HTTPS connection to a host which uses an expired certificate, do not disable certificate verifications entirely. Doing that enables e.g. man in the middle attacks.
If you accept only a single expired and known certificate, you are much less in trouble.

Setup

All the solutions described below use a verify_callback for the request's OpenSSL::X509::Store where you can specify a lambda to adjust its verification response.
Your callback must return either true or false and OpenSSL's verification result is...

List of handy Ruby scripts to transcode different file types (often by using GPT)

It's 2024 and we have tools like ffmpeg, imagemagick and GPT readily available. With them, it's easy to convert texts, images, audio and video clips into each other.

For the everyday use without any parameter tweaking I'm using a collection of tiny scripts in my ~/bin folder that can then be used as bash functions. And: It's faster to use the CLI than interacting with a website and cheaper to use the API than buying GPT plus.. :-)

Usage

  • text-to-image "parmiggiano cheese wedding cake, digital art"
  • `text-to-audio "Yesterday I ate ...

Ruby on Rails: Finding a memory leak

The author describes his little journey in hunting down a memory leak. Maybe his approach and tooling may one day help you in a similar situation.

Tools: rbtrace, ObjectSpace.trace_object_allocations_start, heapy, sheap

Use <input type="number"> for numeric form fields

Any form fields where users enter numbers should be an <input type="number">.

Numeric inputs have several benefits over <input type="text">:

  • On mobile or tablet devices, number fields show a special virtual keyboard that shows mostly digit buttons.
  • Decimal values will be formatted using the user's language settings.
    For example, German users will see 1,23 for <input type="number" value="1.23">.
  • Values in the JavaScript API or when submitting forms to the server will always use a point as decimal separator (i.e. "1.23" eve...

Bash: How to count and sort requests by IP from the access logs

Example

87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.41 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"

Goal

Count and sort the number of requests for a single IP address.

Bash Command

awk '{ print $1}' test.log | sort...

Rails: Testing the number of database queries

There are a few tools to combat the dreaded n+1 queries. The bullet gem notifies you of missing eager-loading, and also if there is too much eager-loading. strict_loading in Rails 6.1+ forces developers to explicitly load associations on individual records, for a single association, for an entire model, or globally for all models.

But you can also actually **write spe...

When loading Yaml contents in Ruby, use the :freeze argument to deep-freeze everything

Ruby methods which load from a Yaml file, like YAML.safe_load or YAML.safe_load_file, support passing freeze: true to deep-freeze the entire contents from the Yaml file.
This is available by default on Ruby 3.0 and newer. On older Rubies, you can install psych 3.2.0 or newer for :freeze support.

As an example, consider the following Yaml file:

---
message:
  - hello
  - universe
foo:
  bar:
    baz: "example"

We can now load it as usual, but pass freeze: true.

>> test = YAML.safe_load_file('example.yml', fre...

Maintaining custom application tasks in Rails

Here are some hints on best practices to maintain your tasks in larger projects.

Rake Tasks vs. Scripts

  • The Rails default is using rake tasks for your application tasks. These live in lib/tasks/*.
  • In case you want to avoid rake for your tasks and just use plain ruby scripts, consider lib/scripts/* as folder.

Keeping tasks slim

For readability and testing it's easier to keep your tasks slim. We suggest to use folders inside the tasks or scripts folder.

Example for a task:

The slim task lib/tasks/gitlab.rb:

Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)

General

A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch

This card deals with specifics concerning the use of searchkick.

Step 1: Make Opensearch available for Searchkick

In your Gemfile

# Search
gem 'searchkick'                   # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'

in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:

SEARCHKICK_CLIENT_T...

OpenAI TTS: How to generate audio samples with more than 4096 characters

OpenAI is currently limiting the Audio generating API endpoint to text bodies with a maximum of 4096 characters.
You can work around that limit by splitting the text into smaller fragments and stitch together the resulting mp3 files with a CLI tool like mp3wrap or ffmpeg.

Example Ruby Implementation

Usage

input_text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Mi eget mauris pharetra et ultrices neque."
output_mp3_path = Rails.root.join("tts/ipsum...

Livereload + esbuild

Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.

We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets and compiles to public/assets; if not change the paths below accordingly.

Basic idea

We will

  • use the guard-livereload gem as the livereload server (which send updates to the browser),
  • use the livereload-js npm package in the browser to con...