MySQL replication how-to

This may be awkward to set up, but will work once you're done.

Fun facts:

  • In case of a connection loss the slave will try to reconnect to the master server and resume replication for the next 24 hours
  • If you want to use your slave as a "real" MySQL server, you basically need to switch off replication (STOP SLAVE; RESET SLAVE; and reset your my.cnf) and restart the MySQL daemon.

Master server configuration

  • Create replication user
    In the MySQL shell:
    CREATE USER 'replicator'@'%' IDENTI...
    

Collection of Rails development boosting frameworks

Development environment setup

Rails Composer

Basically a comprehensive Rails Template. Prepares your development environment and lets you select web server, template engine, unit and integration testing frameworks and more.

Generate an app in minutes using an application template. With all the options you want!

Code generators

Rails Bricks

A command line wizard. Once you get it running, it creates sleek applications.

RailsBricks enables you to cre...

Using the ActiveSupport::BroadcastLogger

The ActiveSupport::BroadcastLogger allows you to log to multiple sinks. You know this behavior from from the rails server command, that both logs to standard out and the log/development.log file.

Here is an example from the ActiveSupport::BroadcastLogger API:

stdout_logger = ActiveSupport::Logger.new(STDOUT)
file_logger = ActiveSupport::Logger.new("development.log")
broadcast = ActiveSupport::BroadcastLogger.new(stdout_logger, file_logger)

broadcast.i...

How to monitor Sidekiq: A working example

In order to have monitoring for Sidekiq (like queue sizes, last run of Sidekiq) your application should have a monitoring route which returns a json looking like this:

{
  "sidekiq": {
    "totals": {
      "failed": 343938,
      "processed": 117649167
    },
    "recent_history": {
      "failed": {
        "2016-11-06": 1,
        "2016-11-07": 46,
        "2016-11-08": 0,
        "2016-11-09": 0,
        "2016-11-10": 0
      },
      "processed": {
        "2016-11-06": 230653,
        "2016-11-07": 230701,
        "2016-11-08"...

Manage Linux services on the command line (Ubuntu)

Ubuntu 18.04 uses systemd to manage services.

There are basically two commands for listing all services and manipulating the state of a certain service: service and systemctl:

  • service manages System V init scripts
  • systemctl controls the state of the systemd system and service manager. It is backwards compatible to System V and includes the System V services

Therefore I prefer to use systemctl.


See which services are there

>systemctl list-units -a --type=service
  UNIT                                 LOAD     ...

How to open files from better_errors with RubyMine on Linux

I recently noticed that better_errors allows you to to open files from within your favorite editor. However it was not so easy to get rubymine:// links to work on Gnome/Linux. Here is how it finally worked for me:

Step 1: Add a Desktop launcher

Add this file to ~/.local/share/applications/rubymine.desktop:

[Desktop Entry]
Version=1.0
T...

Some tips for upgrading Bootstrap from 3 to 4

Recently I made an upgrade from Bootstrap 3 to Bootstrap 4 in a bigger project. Here are some tips how to plan and perform such an upgrade. The effort will scale with the size of the project and its structure. If your stylesheets already follow strict rules, it may take less time to adapt them to the new version.

Preparation

There are several gems and libraries that works well with bootstrap or provide at least stylesheets/plugins to easily integrate the bootstrap theme. But very often they only work with specific version or are no long...

Upgrading from Capistrano 2 to 3

Capistrano 3 is a major rework of the framework and requires several adjustments to your deploy configuration files. The biggest change is that they moved away from their custom DSL and use Rake instead. For connecting with and operating on the servers, they bring a new gem SSHKit which does the heavy lifting. It's SSHKit's DSL that is used anywhere inside the Rake tasks. See #Resources at the bottom for examples.

Step 1: Upgrade guide

For migration from 2 to 3, follow this tutorial: [Capistrano 3 Upgrade Guide](https://semaphorec...

Capistrano: Configure environment specific array attributes

Using Capistrano, we usually have some array configurations in the config/deploy.rb file, like set :linked_files, %w[config/database.yml], so in this case we don't have to manage the database configuration individually on every server.

In a specific case, one of our projects supports sign in by SAML, but not every deploy target has this feature activated. Here comes a nice handy Capistrano feature, which lets us modify the default configuration for individual env...

Version 5 of the Ruby Redis gem removes Redis.current

Redis.current will be removed without replacement in redis-rb 5.0.
Version 4.6.0 adds deprecation warnings for Redis.current and Redis.current=:

`Redis.current=` is deprecated and will be removed in 5.0.

If your application still uses Redis.current, you can only fix it by no longer using it. Here is how.

Redis.new when you need it

You can easily instantiate a Redis client when you need it.

There is probably already a constant like REDIS_URL that you use to configure Sidekiq or similar. So just use that one.

``...

Setup Sidekiq and Redis

If you want Sidekiq to be able to talk to Redis on staging and production servers, you need to add the following to your configuration:

# config/initializers/sidekiq.rb
require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { url: REDIS_URL }
end

Sidekiq.configure_server do |config|
  config.redis = { url: REDIS_URL }
end

The following step may be skipped for new Sidekiq 6+, since it isn't recommended anymore to use a global redis client.

# config/initializers/redis.rb
require 'redis'
require_relativ...

request_store: Per-request global storage for your Rails app

Ever needed to use a global variable in Rails? Ugh, that's the worst. If you need global state, you've probably reached for Thread.current.

When you're using Thread.current, you must make sure you're cleaning up after yourself. Else, values stored in one request may be available to the next (depending on your server). request_store wipes all data when a request ends and makes per-request global storage a no-brainer. Internally, it's using Thread.current with a Hash in a simple middleware.

Example: Remembering all currently a...

Capistrano: creating a database dump if migrating

In Capistrano 3, your Capfile requires 'capistrano/rails/migrations', which brings two Capistrano tasks: deploy:migrate and deploy:migrating. The former checks whether migrations should be performed. If so, the latter is invoked, which performs the actual migrations.

Knowing this, it is easy to dump the db only if migrations will run. First, enable conditional migrations:

# config/deploy.rb
set :conditionally_migrate, true # Only attempt migration if db/migrate changed

Then hook up the dump task to deploy:migrating:

Get rid of WARNING: Nokogiri was built against LibXML version 2.7.7, but has dynamically loaded 2.7.8

If you get this warning on your local machine one of these steps might help:

Rebuilt the gem with the newer library

gem install --no-rdoc --no-ri nokogiri -- --with-xml2-include=/opt/local/include/libxml2 --with-xml2-lib=/opt/local/lib

If you still get the error, try to uninstall all nokogiri versions with

gem uninstall nokogiri

and install nokogiri again.

Fixing the issue on servers

However, on our servers this probably will not work. On the server, gems are stored in the ../shared/bundle/ruby/:version/gems dire...

How Rails and MySQL are handling time zones

When working with times and dates in Rails applications, you need to deal with the following problem:

  • In Rails, Time objects have a time zone. You can get the zone name by doing time_object.zone.
  • This zone is considered when doing time calculations, e.g. 10 AM CEST minus 8 AM UTC is zero.
  • A datetime in MySQL does not have a zone. It just stores the literal string "2010-05-01 12:00:00".
  • That means that Rails must make assumptions about timestamps loaded from and written to MySQL.

Rails has two completely different modes ...

Reverse-proxying web applications with Apache 2.4+

Note: Making a reverse proxy with nginx is much more straightforward.


A reverse proxy is a "man in the middle" server that tunnels requests to another server. You can use for things like:

  • Expose a local service that you cannot directly reach over the internet
  • "Change" the domain or path of a web application by rewriting them on the fly
  • Instantly change servers that respond to a name or ...

Make Capistrano use SSH Key Forwarding

When deploying code with Capistrano (depending on your configuration) at some point Capistrano tries to check out code from your repository. In order to do so, Capistrano connects to your repository server from the application server you're deploying to with SSH. For this connection you can use two SSH keys:

  • the user's ~/.ssh/id_rsa [default]
  • the very same key you used for connecting to the application server - forwarded automatically to the git repository.

If you prefer the second way, add this to deploy.rb:

ssh_options[:forwar...

Rails: Rest API post-mortem analysis

This is a personal post-mortem analysis of a project that was mainly build to provide a REST API to mobile clients.

For the API backend we used the following components:

  • Active Model Serializer (AMS) to serializer our Active Record models to JSON.
  • JSON Schema to test the responses of our server.
  • SwaggerUI to document the API.

It worked

The concept worked really good. Here are two points that were extraordinary compared to normal Rails project with many UI components:

  • Having a Rails application, that has no UI components (only...

Five years of "Today I Learned" from Josh Branchaud

The linked GitHub repository is a bit like our "dev" cards deck, but groomed from a single person (Josh Branchaud). It includes an extensive list of over 900 TILs on many topics that might be interesting for most of us. (e.g. Ruby, Rails, Git, Unix..)

Ruby

Here is an excerpt of all the Ruby TILs that were new to me. I encourage you to take your time to skim over the original list as well!

Rails: render a template that accepts a block by using the layout option of render

Let's say you have a form that you render a few times but you would like to customize your submit section each time. You can achieve this by rendering your form partial as layout and passing in a block. Your template or partial then serves as the surrounding layout of the block that you pass in. You can then yield back the form to the block and access the form in your block.

-# record/_form.haml

= form_for record do |form|
  -# ...
  .form-actions
    yield(form)
  

In order to make your template record/_form.haml accept a...

How to fix parallel_tests with Redis on powerful machines

When you have a powerful machine with many CPU cores, you might run into an error like

ERR DB index is out of range (Redis::CommandError)

This is because Redis defaults to at most 16 databases (0 to 15) and running tests in parallel might exceed that (your tests might run on databases 1..n or 2..(n+1)).

You can increase that limit:

  1. Get number of CPUs of your machine.

    nproc --all
    
  2. Open up Redis configuration file.

    sudo vim /etc/redis/redis.conf
    
  3. Find databases row and increase it, e.g. set to 32:

...

How to downgrade Google Chrome in Ubuntu

If you're experiencing problems with your Google Chrome installation after an update, it might help downgrading Chrome to check if the problem disappears. Keep in mind though that running outdated software, especially web browsers, is in most cases not a good idea. Please verify periodically if you still need to run the old version or if an even more recently updated version fixes the problems introduced in your version.

Here's how to get old versions of Chrome for your Ubuntu installation:

First, go to [UbuntuUpdates](https://www.ubuntuup...

Mock the browser time or time zone in Selenium features

In Selenium features the server and client are running in separate processes. Therefore, when mocking time with a tool like Timecop, the browser controlled by Selenium will still see the unmocked system time.

timemachine.js allows you to mock the client's time by monkey-patching into Javascript core classes. We use timemachine.js in combination with the Timecop gem to synchronize the local browser time to the ...

Debugging your Webpack build time with Speed Measure Plugin

If your Webpack build is slow, you can use the Speed Measure Plugin for Webpack to figure out where time is being spent.
Note that at time of writing, Webpack 5 seems unsupported. It works on Webpack 4, though.

Wire it into your application as described in the library's documentation:

  1. Hook into your environment file, e.g. config/webpack/development.js and instead of exporting your Webpack config,...