Vortrag: Elasticsearch Grundlagen und Rails-Integration mit searchkick

Was ist Elastic?

  • Suchmaschine, basierend auf Apache Lucene
  • größtenteils Open-Source
  • einige kommerzielle Features ("Elastic Stack", früher "X-Pack")
    • Zugriffsrechte (bis vor kurzen)
    • Monitoring
    • Reporting
    • Graph-Unterstützung
    • Machine Learning
  • REST-Api (JSON über HTTP)

Grundlagen

Elastic antwortet per Default auf Port 9200

http GET :9200
{
  "name": "ntK2ZrY",
  "cluster_name": "elasticsearch",
  "cluster_uuid": "Bbc-ix5bQZij5vfFU29-Cw",
  "version": {
    "number": "6.7.1",
    "build_flavor": "...

Tod: A Gem for handling daytime without a date

Tod is a gem for working with daytimes. That's a tuple of (hour, minute second) without a day, month or year.

Another additional gem?

Thus SQL has a time datatype for storing time of day in the format hh:mm:ss, neither Ruby nor Rails themselves offer an elegant way to deal with day times.

Time and DateTime both handle daytime values AND calendar date, using them to only store the time of day will end in inconsistent and thus confusing data, e. g. Time.new will initialize with the current Time in your Timezone, `DateTime.n...

When reading model columns during class definition, you must handle a missing/empty database

When doing some meta-programming magic and you want to do something for all attributes of a class, you may need to access connection or some of its methods (e.g. columns) during class definition.

While everything will be fine while you are working on a project that is in active development, the application will fail to boot when the database is missing or has no tables. This means that Raketasks like db:create or db:migrate fail on a freshly cloned project.

The reason is your environment.rb which is loaded for Raketasks and calls...

Migration from the Asset Pipeline to Webpacker

This is a short overview of things that are required to upgrade a project from the Asset Pipeline to Webpacker. Expect this upgrade to take a few days even the diff is quite small afterwards.

Preparations

1. Find all libraries that are bundled with the asset pipeline. You can check the application.js and the application.css for require and import statements. The source of a library is most often a gem or a vendor directory.
2. Find an working example for each library in the application and write it down.
3. Find out the ver...

ActionMailer: Previewing mails directly in your email client

In Rails, we usually have a mailer setup like this:

class MyMailer < ActionMailer::Base

  def newsletter
    mail to: 'receiver@host.tld',
      from: 'sender@host.tld',
      subject: 'My mail'
  end

end

If you want to preview your mail in the browser, you can use the Action Mailer Preview. To inspect the mail directly in your email client, just create an .eml file and open it with your client:

mail = MyMailer.newsletter
Fil...

Why you can't use timezone codes like "PST" or "BST" for Time objects

Rails' ActiveSupport::TimeWithZone objects have both a timezone code and offset, e.g. Thu, 28 Mar 2019 16:00:00 CET +01:00. Ruby's stdlib TZInfo also has time zones, but with different identifiers.
Unfortunately, not all timezone codes can be used to parse strings or to move time objects into another time zone.

Some timezone codes like CET are supported by ActiveSupport extensions like String#in_time_zone, while many codes will actually not work:

>> '2019-03-01 12:00'.in_time_zone('PST')
ArgumentError (Invalid Timezone: PST)
...

rails_state_machine 1.1.2 released

Fixed bugs where calling valid? would change the record's state.

How to fix parallel_tests with Redis on powerful machines

When you have a powerful machine with many CPU cores, you might run into an error like

ERR DB index is out of range (Redis::CommandError)

This is because Redis defaults to at most 16 databases (0 to 15) and running tests in parallel might exceed that (your tests might run on databases 1..n or 2..(n+1)).

You can increase that limit:

  1. Get number of CPUs of your machine.

    nproc --all
    
  2. Open up Redis configuration file.

    sudo vim /etc/redis/redis.conf
    
  3. Find databases row and increase it, e.g. set to 32:

...

Shortcuts for getting ids for an ActiveRecord scope

You can use .ids on an ActiveRecord scope to pluck all the ids of the relation:

# Modern Rails
User.where("users.name LIKE 'Foo Bar'").ids

# Rails 3.2+ equivalent
User.where("users.name LIKE 'Foo Bar'").pluck(:id)

# Edge rider equivalent for Rails 2+
User.where("users.name LIKE 'Foo Bar'").collect_ids

Upgrade guide for moving a Rails app from Webpack 3 to Webpack 4

Webpacker is Rails' way of integrating Webpack, and version 4 has been released just a few days ago, allowing us to use Webpack 4.

I successfully upgraded an existing real-world Webpack 3 application. Below are notes on everything that I encountered.
Note that we prefer not using the Rails asset pipeline at all and serving all assets through Webpack for the sake of consistency.

Preparations

  • Remove version locks in Gemfile for webpacker
  • Remove version locks in package.json for webpack and webpack-dev-server
  • Install by ca...

Rails: How to get URL params without routing parameters (or vice versa)

Rails' params hash contains any request parameters (URL parameters or request payload) as well as routing parameters like :controller, :action, or :id.

To access only URL parameters, use request.query_parameters. Routing params are available through request.path_parameters.

# On /users?query=Bob&page=2

>> request.params
=> {"page"=>"2", "query"=>"Bob", "controller"=>"users", "action"=>"index"}

>> request.query_parameters
=> {"page"=>"2", "query"=>"Bob"}

>> request.path_parameters
=> {:controller=>"users", :action=>"i...

Carrierwave processing facts

  1. Class-level process definitions are only applied to the original file
  2. Versions are generated based on the processed original file
  3. Callbacks (before/after) are applied to original file and each version by itself
  4. Under the hood, a version is an instance of the uploader class that has no versions
  5. Version uploader and original uploader can be distinguished by checking #version_name: version uploaders return the version name, whereas the original uploader instance returns nil
  6. Version instances do not have a re...

Trigram indexing as an alternative to PostgreSQL fulltext search

For searching in large database tables we usually use PostgreSQL's fulltext search capabilities.

While this works reasonably well for content primarily consisting of prose, it is not necessarily a good solution for all use cases. The main issue is that it is only possible to search for prefixes of text tokens, which can potentially be unexpected for users.

One example are dates:

If you index the text 2019-01-23 15:16, PostgreSQL will create the following tokens: 2019, -01, -23, 15 16. A user searching for 01-23 wi...

JavaScript without jQuery

This is a presentation from 2019-01-21.

Summary

  • We want to move away from jQuery in future projects
  • Motivations are performance, bundle size and general trends for the web platform.
  • The native DOM API is much nicer than it used to be, and we can polyfill the missing pieces
  • Unpoly 0.60.0 works with or without jQuery

Is jQuery slow?

From: Sven
To: unpoly@googlegroups.com
Subject: performance on smartphones and tablets

Hello

I just used your framework in one project and must say,
I am really pleased with it -- but o...

Heads up: pg_restore --clean keeps existing tables

When restoring a PostgreSQL dump using pg_restore, you usually add the --clean flag to remove any existing data from tables.

Note that this only removes data from tables that are part of the dump and will not remove any extra tables. You need to do that yourself.

When is this relevant?

As an example: You want to load a staging dump into your development machine. On your development machine, you have run migrations that introduced more tables which do not yet exist on staging. pg_restore with --clean will loa...

How to drop all tables in PostgreSQL

To remove all tables from a database (but keep the database itself), you have two options.

Option 1: Drop the entire schema

You will need to re-create the schema and its permissions. This is usually good enough for development machines only.

DROP SCHEMA public CASCADE;
CREATE SCHEMA public;

GRANT ALL ON SCHEMA public TO postgres;
GRANT ALL ON SCHEMA public TO public;

Applications usually use the "public" schema. You may encounter other schema names when working with a (legacy) application's database.

Note that f...

How to upgrade Rails: Workflow advice

When upgrading Rails versions -- especially major versions -- you will run into a lot of unique issues, depending on the exact version, and depending on your app.

However, it is still possible to give some generic advice on how you want to tackle the update in principle.

If you are not really confident about upgrading Rails, have a look at Rails LTS.

How many update steps?

Besides the Rails upgrade itself, you might also want to upgrade your other gems and upgrade your Ruby version.
First decide in how many st...

Regular tasks for long-running projects

When projects run for many years, they require special regular maintenance to stay fresh. This kind of maintenance is usually not necessary for small or quick projects, but required to keep long-running projects from getting stale.

You should be able to fit this into a regular development block.

Quarterly

Check which libraries need updating

As time goes by, libraries outdate. Check your software components and decide if any of it needs an update. Your main components (e.g. Ruby, Rails, Unpoly) should always be reasonably up to da...

Rubymine FileType mismatch

If your Rubymine does not recognize a file type correctly although you have entered the unmistakeable file extension like material_orders_controller.rb, this may help you:

Causing the Problem

Sometimes you create a new file and forget to enter the ending like material_orders_controller
Rubymine handles such files per default as simple txt files.
You delete this file and create a new one with correct ending: material_orders_controller.rb. But still Rubymine treats this file as text file, no highlighting is available.

What happene...

ActiveJob Inline can break the autoloading in development

We figured out, that ActiveJob Inline might lead to autoloading problems in development. The result was an exception when running an import script, which delivers async mails.

A copy of XXX has been removed from the module tree but is still active! (ArgumentError)

Our fix was to use .deliver_now and not .deliver_later (this is not a general fix, but it was okey for us). Below there are some debug hints which helped us to locate the problem:

  • We placed a pry debugger in [ActiveSupport#clear](https://github.com/rails/rails/blob...

Devise: Don't forget to lock users with soft delete

There are two ways to lock a user in devise.

  1. Using the lockable module
  2. Customizing the user account status validation when logging in.

It depends on your requirements which methods works best.

Locking a user on soft delete

We recommend to use option 2 when you want to couple the lock to the m...

RSpec 3 allows chaining multiple expectations

When you are using lambdas in RSpec to assert certain changes of a call, you know this syntax:

expect { playlist.destroy }.to change { Playlist.count }.by(-1)

While you can define multiple assertions through multiple specs, you may not want to do so, e.g. for performance or for the sake of mental overhead.

Multiple expectations on the same subject

RSpec allows chaining expectations simply by using and.

expect { playlist.destroy }
  .to change { Playlist.count }.by(-1)
  .and not_change { Video.count }

...

cucumber_factory 1.14 lets you set array fields, has_many associations, numbers without quotes

Setting array columns

When using PostgreSQL array columns, you can set an array attribute to a value with square brackets:

Given there is a movie with the tags ["comedy", "drama" and "action"]

Setting has_many associations

You can set has_many associations by referring to multiple named records in square brackets:

Given there is a movie with the title "Sunshine"
And there is a movie with the title "...

Minify Font Awesome fonts with webpack

Font Awesome 5 is a comprehensive solution for vector icons on your website.

Originally, Font Awesome came as an icon font (plus stylesheets), but recently it can also be used as a pure JavaScript solution (which will render icons as inline <svg> tags), or even as SVG sprites.

All solutions have their pros and cons:

Icon font:

  • little CPU load (no JavaScript)
  • fonts are relatively large
  • 1 extra HTTP request

Javascript + inline SVG:

  • higher CPU load (needs to watch the DOM via mutation observers to ad...