Storing trees in databases

This card compares patterns to store trees in a relation database like MySQL or PostgreSQL. Implementation examples are for the ActiveRecord ORM used with Ruby on Rails, but the techniques can be implemented in any language or framework.

We will be using this example tree (from the acts_as_nested_set docs):

root
|
+-- Child 1
|   |
|   +-- Child 1.1
|   |
|   +-- Child 1.2
|
+-- ...

Custom RSpec matcher for allowed values (or assignable_values)

In contrast to RSpec's included allow_value matcher, the attached matcher will also work on associations, which makes it ideal for testing assignable_values.

Usage example

describe Unit do

  describe '#building' do
    it 'should only allow buildings that a user has access to' do
      building = build(:building)
      other_building = build(:building)
      unauthorized_building = build(:building)

      power = Power.new(build(:user))

      Power.with_power(power) do
        expect(power).to receive(:buildings).at_least...

Beware when using ActiveSupport time and date calculation methods

The pitfall

Rails Active Support provides some helpful methods for calculating times and dates, like Duration#ago or Duration#from_now. But beware when using those, because they wont give you Dates or Times but ActiveSupport::TimeWithZone instances. As the class name hints, you now have to be awa...

Rails: Overriding view templates under certain conditions only

Rails offers a way to prepend (or append) view paths for the current request. This way, you can make the application use different view templates for just that request.

Example

A use case of this is a different set of view templates that should be used under certain circumstances:

class UsersController < ApplicationController

  before_action :prepare_views
  
  def index
    # ...
  end    
  
  private
  
  def prepare_views
    if <condition>
      prepend_view_path Rails.root.join('app', 'views', 'special')
    end
  end
 ...

Pretty commit messages via geordi

Geordi provides a pretty neat way to generate beautiful commit messages according to your stories in Linear:

geordi commit

Geordi reads from a .geordi.yml file inside your repo and connects to Linear to list started and finished stories with their title. Choosing one of them generates a commit message including id and title from Linear app and a link to the original issue. For example:

[VW-1337] CRUD Users

Issue: https://linear.app/makandra/issue/VW-1337/crud-user...

A quick introduction to CORS

Background

Cross-Site Request Forgery (CSRF) is an attack pattern for websites. A CSRF attack is usually relevant in a browser context, where state is kept for multiple domains (as opposed to independent requests made e.g. with curl). The most common example is authentication via cookies. If a script on https://example.com made requests to https://docs.google.com, the browser would send all cookies for docs.google.com along, effectively given the script access to anythin...

Rails: Composing an ETag from multiple records

Rails offers the fresh_when method to automatically compute an ETag from the given record, array of records or scope of records:

class UsersController < ApplicationController
  def show
    @user = User.find(params[:id])
    fresh_when @user
  end  
  
  
  def index
    @users = User.all.to_a
    fresh_when @users
  end
end

When your view also displays other records (typically associations), those other records should be included in the ETag. You can do so by passing an array of ETaggable objects to fresh_when.

...

Rails: Your index actions probably want strict_loading

By activating strict_loading you force developers to address n+1 queries by preloading all associations used in the index view. Using an association that is not preloaded will raise an ActiveRecord::StrictLoadingViolationError.

I think it's a good default to activate strict_loading in your controllers' #index actions. This way, when a change introduces an n+1 query, you...

Rails: How to test the parsed response body

Testing your responses in Rails allows to parse the body depending on the response MIME type with parsed_body.

get '/posts.json'
response.parsed_body # => [{'id' => 42,  'title' => 'Title'}, ...]

For JSON APIs we often parse the response as symbolized keys with JSON.parse(response.body, symbolize_names: true), which is not supported by parsed_body. For all other cases you might want to drop JSON.parse(response.body) and replace it w...

Best practices: Large data migrations from legacy systems

Migrating data from a legacy into a new system can be a surprisingly large undertaking. We have done this a few times. While there are significant differences from project to project, we do have a list of general suggestions.

Before you start, talk to someone who has done it before, and read the following hints:

Understand the old system

Before any technical considerations, you need to understand the old system as best as possible. If feasible, do not only look at its API, or database, or frontend, but let a user of the old system sho...

Keeping web applications fast

Our applications not only need to be functional, they need to be fast.

But, to quote Donald Knuth,

premature optimization is the root of all evil (or at least most of it) in programming

The reasoning is that you should not waste your time optimizing code where it does not even matter. However, I believe there are some kinds of optimizations you should do right away, because

  • they are either obvious and easy
  • or they are very hard to do optimize later

This is an attempt to list some of those things:

On the server

...

How to exclusively lock file access in ruby

We will achieve this by creating a block accepting method to optionally create and then lock a .lock File of the underlying accessed file.

Why create a .lock file?

  • The main advantage of creating a .lock file is that #flock might block some operations and require the index node of the file to be consistent. Some operations might change that index node.
  • In some cases it might also be convenient to just read/write the lock file first and update the other file afterwards or vice versa, such that breaking of a process does not...

Rails developers: Have better context in Git diffs

Git diffs show the surrounding contexts for diff hunks. It does so by applying regular expressions to find the beginning of a context. When it comes to Ruby, however, it will not find method heads and travel up to the class definition:

@@ -24,7 +24,7 @@ class TicketPdf # <=== Actually expected here: the method definition
     ApplicationController.render(
       "tickets/index.html.haml",
       layout: "tickets",
-      assigns: { tickets: tickets }
+      assigns: { tickets: tickets, event_name: event_name }
     )
   end
 end
```...

Git: Improve your commits by reviewing changes one-by-one

Git commits should be very deliberate, and only contain changes that you really want to be in there. In order to reduce the chance to accidentally commit something you didn't intend, review your changes before committing.

My preferred way of doing this is (only using git)

git add -N . # Add all paths, but not their contents
git add -p

Git will now show you all your changes in small chunks and ask you in an interactive mode whether you really want to add them.

The most helpful commands are

  • y: yes (add the change)
  • ...

Project management best practices: Project team responsibilities

In a project team for a bigger project people have several roles:

  • Developer: at makandra
  • Project lead: at makandra
  • Project manager (PM): at makandra, external, or with the customer. In a smaller project this person is also the project lead.
  • Product owner (PO): with the customer

Developer

  • Development
  • Take responsibility for their stories. This includes always gathering all necessary information from the project lead or the PM, communicate blockers, make sure stories are merged, deployed etc.
  • Tell the project lead, if y...

SASS: Reusing styles from other files

SASS has an @extend keyword to inherit styles.

.alert
  color: red
  
  &.-framed
    border: 1px solid red
    padding: 5px
    
  &.-homepage
    @extend .-framed
    border-width: 5px

When compiling, SASS will simply join the selectors. Note how .-homepage is written where .-framed was defined:

...
.alert.-framed, .alert.-homepage {
  border: 1px solid red;
  padding: 5px;
}
.alert.-homepage {
  border-width: 5px;
}

Warning

Unfortunately, this does...

How to enable template coverage support for simplecov

Since Ruby 3.2.0 you can measure coverage support for eval statements and support has been added for the simplecov gem as well.

This allows to track coverage across ruby templates such as haml, erb, ...

Simply set this within simplecov

SimpleCov.start do
  enable_coverage_for_eval
end

SVGO: SVG Optimizer

SVG files are often much larger than necessary, containing comments, metadata, hidden elements etc.

Optimize them with this tool.

Note that for a properly scaling SVG, you need to keep the viewBox attribute. There's an option --disable=removeViewBox for this.

FactoryBot: Passing attributes to associated records using transient attributes

FactoryBot.define do

  factory :parent do
    transient do
      child_name nil
      child_allowed_to_drive false
    end
    
    child do
      association(:child, name: child_name, allowed_to_drive: child_allowed_to_drive)
    end
  end

  factory :child do
    name 'Child'
    allowed_to_drive false
  end

end

# Usage
daughter = FactoryBot.create(:parent, child_name: 'Lisa').child
daughter.name # => 'Lisa'
daughter.allowed_to_drive? # => false

son = FactoryBot.create(:parent, child_name: 'Benedikt', child_allowed_to_drive: tr...

Project management best practices: User stories & Issues

We organize our daily work with issues in our Linear workspace.

Issue format

A good issue needs to be precise. It should be very clear what is part of an issue, and what is not. If there are different expectations between the person who writes and who implements an issue, there will be rejects.

To this end, we use a consistent format for issues that looks like this:

Issue: Autocomplete

As a journalist, I want to have an autocomplete in the search bar, to have a more efficient way to find articles.

Acceptance criteri...

Deterministic ordering of records by created_at timestamp

Creating records in specs can be so fast that two records created instantly after one another might have the same created_at timestamp (especially since those timestamps don't have an indefinitely high resolution). When ordering lists by timestamps, you should therefore always include a final order condition using the primary key of the table.

class Photo < ActiveRecord::Base
  scope :by_date, -> { order('created_at DESC, id DESC') }
end

Photo.by_date

Remember to include the id field in the database index.

Rails migration: Changing a column type without losing the content

The change_column method for rails migrations support casting with a custom SQL statement. This allows us to change a column type and keep the former content as the new type. This way, we can for example prepare an address number column to hold German address numbers, which can contain letters:

Example (in most cases not a good idea!)

class ChangeAnIntegerColumnToString < ActiveRecord::Migration[6.1]
  def up
    change_column :users, :address_number, 'varchar USING CAST(rating AS varchar)'
  end

  def down
    change_column ...

CSS: CSS Container Queries

Container queries enable you to apply styles to an element based on the size of the element's container. If, for example, a container has less space available in the surrounding context, you can hide certain elements or use smaller fonts. Container queries are an alternative to media queries, which apply styles to elements based on viewport size or other device characteristics.

This feature is now stable across browsers.

Warning

This feature landed in browsers in the beginning of 2023. According to our support policy this will bec...

Creating a sample video with ffmpeg

If you need a sample video with certain properties for a test you can create one using ffmpeg.
You might want a very low bitrate file to speed up processing in your test. (e.g. you only care about the length, then you can create a video with a very low resolution and framerate)

Create a 21s video with 1fps and 10x10 resolution:
ffmpeg -t 21 -s 10x10 -r 1 -f rawvideo -pix_fmt rgb24 -i /dev/zero sample_21_seconds.mp4

Option Explanation
-t 21 set the length to 21s
-s 10x10 set the resolution the 10 by 10 p...