Rails: Concurrent requests in development and tests

With puma you can have concurrent requests. There are two concepts on how Puma can handle two incoming requests: Workers and Threads.

Workers

Puma can have multiple workers. Each worker is a process fork from puma and therefore a very heavy instance and can have multiple threads, that handle the incoming requests.

Example: A Puma server with 2 workers and 1 thread each can handle 2 request in parallel. A third request has to wait until the thread of one of the workers is free.

Threads

Rails is thread-safe since version 4 (n...

Bundler in deploy mode shares gems between patch-level Ruby versions

A recent patch level Ruby update caused troubles to some of us as applications started to complain about incompatible gem versions. I'll try to explain how the faulty state most likely is achieved and how to fix it.

Theory

When you deploy a new Ruby version with capistrano-opscomplete, it will take care of a few things:

  • The new Ruby version is installed
  • The Bundler version stated in the Gemfil...

How to: Throttle CPU in Google Chrome

Chrome allows you to throttle the Network and the CPU. Both settings are useful to measure the performance of you application and reproduce performance issues (Example Debugging frontend performance issues with Chrome DevTools).

You find the settings in: DevTools > Performance > Capture Settings (Gear icon in the right corner) > `CPU: No...

ActiveRecord: Specifying conditions on an associated table

We can use ActiveRecord's where to add conditions to a relation. But sometimes our condition is not on the model itself, but on an associated model. This card explains multiple ways to express this condition using ActiveRecord's query interface (without writing SQL).

As an example we will use a User that has many Posts:

class User < ApplicationRecord
  has_many :posts
  scope :active, -> { tra...

How to write modular code

Or: How to avoid and refactor spaghetti code

Please note that I tried to keep the examples small. The effects of the methods in this card are of course much more significant with real / more complex code.

What are the benefits of more modular code?

Code is written once but read often (by your future self and other developers who have to understand it in order to make changes for example). With more modular code you reduce the scope of what has to be understood in order to change something. Also, naming things gives you the opportunity t...

Rails: Rest API post-mortem analysis

This is a personal post-mortem analysis of a project that was mainly build to provide a REST API to mobile clients.

For the API backend we used the following components:

  • Active Model Serializer (AMS) to serializer our Active Record models to JSON.
  • JSON Schema to test the responses of our server.
  • SwaggerUI to document the API.

It worked

The concept worked really good. Here are two points that were extraordinary compared to normal Rails project with many UI components:

  • Having a Rails application, that has no UI components (only...

How to: Validate dynamic attributes / JSON in ActiveRecord

PostgreSQL and ActiveRecord have a good support for storing dynamic attributes (hashes) in columns of type JSONB. But sometimes you are missing some kind of validation or lookup possibility (with plain attributes you can use Active Record's built-in validations and have your schema.rb).

One approach about being more strict with dynamic attributes is to use JSON Schema validations. Here is an example, where a project has the dynamic attributes analytic_stats, that we can use to store analytics from an external measurement tool.

  • A g...

Resources for working with the Youtube API

You can use the Youtube API to get data from youtube. Here are some resources and some things that are good to know.

You need to create an API key and each request will cost a certain amount of Request-Points (depending on the kind of query).

It may help to understand youtube's domain model before starting.

It's best to just try out your requests in the browser to see what you will get.
A request can look like this (note that you have to replace the playlis...

Checking database size by row count

As an application exists, data accumulates. While you'll be loosely monitoring the main models' record count, some supportive database tables may grow unnoticed.

To get a quick overview of database table sizes, you can view the row count like this:

PostgreSQL

SELECT schemaname,relname,n_live_tup 
FROM pg_stat_user_tables 
ORDER BY n_live_tup DESC
LIMIT 12;

 schemaname |                    relname                     | n_live_tup 
------------+------------------------------------------------+------------
 public     | images...

How to enable Chromedriver logging

When using Chrome for Selenium tests, the chromedriver binary will be used to control Chrome. To debug problems that stem from Selenium's Chrome and/or Chromedriver, you might want to enable logging for the chromedriver itself. Here is how.

Option 1: Use Selenium::WebDriver::Service

In your test setup, you may already have something like Capybara::Selenium::Driver.new(@app, browser: :chrome, options: ...), especially when passing options like device emulation.

Similar to options, simply add an extra key service and pass an inst...

Joining PDFs with Linux command line

There are several ways to merge two (or more) PDF files to a single file using the Linux command line.

If you're looking for graphical tools to edit or annotate a PDF, we have a separate card for that.

PDFtk (recommended)

PDFtk is a great toolkit for manipulating PDF documents. You may need to install it first (sudo apt install pdftk).
Merging multiple files works like this:

pdftk one.pdf two.pdf cat output out.pdf

Unlike pdfjam, PDFtk should not mess with page sizes but simply joins pages as they are.

...

Known MariaDB version incompatibilities

MariaDB (and MySQL) is released in different versions with different behaviors. For backwards compatibility this can be managed with the sql_mode option.

Following you can find behavior differences which cannot be changed by this option. This can cause a regression in an existing application. There is no workaround available. The application needs to be adapted.

Queries with aggregated date types

For a database instance of MariaDB-10...

Ruby: How to use global variables for a conditional debugger

You can share a state in Ruby with global variables. Even if you should avoid them whenever possible, for debugging an application this could be temporary quite handy.

Example:

class User

  after_save { byebug if $debug; nil }

  def lock
   self.locked = true
   save
  end

end
Rspec.describe User do

  let(:user) { create(:user) } 

  before do
   # Many users are created and saved in this hook, but we don't want the debugger to stop for them...

Webpack(er): A primer

webpack is a very powerful asset bundler written in node.js to bundle (ES6) JavaScript modules, stylesheets, images, and other assets for consumption in browsers.

Webpacker is a wrapper around webpack that handles integration with Rails.

This is a short introduction.

Installation

If you haven't already, you need to install node.js and Yarn.

Then, put

gem 'webpacker', '~> 4.x' # check if 4.x is still cu...

RubyMine: Efficiently filtering results in the "Finder" overlay

RubyMine comes with a nice way to grep through your project's files: The finder (ctrl + shift + f). Don't be discouraged about the notice 100+ matches in n+ files if your searched keyword is too general or widely used in your project.

Image

RubyMine comes with a few ways to narrow down the resulting list, don't hesitate to apply those filters to speed up your search. Your keybinding might vary based on your personal settings.

File mask (alt + k)

If you already know the file extension of your ...

ActiveType 1.2 supports "change_association"

With ActiveType 1.2 you can modify associations (has_many etc.) after they have been defined.

One common use case for this is to change an association inside a form model, like this:

class Credential < ActiveRecord::Base
end

class User < ActiveRecord::Base
  has_many :credentials
end

class SignUpCredential < ActiveType::Record[Credential]
end

class SignUp < ActiveType::Record[User]
  change_association :credentials, class_name: 'SignUpCredential'
end

Now, if you load `credentials...

Deal with certain travis CI failures

Travis changed their default distribution from Ubuntu 14.04 (trusty) to 16.04 (precise). This might break your test setup for new builds.

You can solve this issue by freezing your test distribution in the .travis.yml to Ubuntu 14.04 until you have the time to solve all the issues you will have in 16.04:


dist: trusty

Error details

Here are few indicators that you ran into this issue:

Connection to the PostgreSQL database does not work anymore

Your travis-ci builds might have started failing on the usual

psql -c...

Documenting your Rails project's Node.js version in .nvmrc

Not all versions of Node.js are compatible with each other. Also npm packages may require a minimum or maximum version of Node.js. We use nvm on our development PCs so we can operate multiple versions of Node.js in parallel.

To make sure that all developers use a compatible version of Node.js, your Rails project should declare the required Node.js in a file called .nvmrc.

When a .nvmrc exists, developers can cd in your project directory and activate...

How to recognize CVE-2019-5418

If you get requests with values for formats like this:

{:locale=>[:de], :formats=>["../../../../../../../../../../etc/services{{"], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :haml]}

or fails like this:

Invalid query parameters: invalid %-encoding (../../../../../../../../../etc/passwd%%0000.html)

Someone tries to exploit CVE-2019-5418.
If you use the latest Rails (or latest Rails LTS) you're...