MySQL replication how-to
This may be awkward to set up, but will work once you're done.
Fun facts:
- In case of a connection loss the slave will try to reconnect to the master server and resume replication for the next 24 hours
- If you want to use your slave as a "real" MySQL server, you basically need to switch off replication (
STOP SLAVE; RESET SLAVE;
and reset your my.cnf) and restart the MySQL daemon.
Master server configuration
-
- Create replication user
- In the MySQL shell:
CREATE USER 'replicator'@'%' IDENTI...
Collection of Rails development boosting frameworks
Development environment setup
- Rails Composer
-
Basically a comprehensive Rails Template. Prepares your development environment and lets you select web server, template engine, unit and integration testing frameworks and more.
Generate an app in minutes using an application template. With all the options you want!
Code generators
- Rails Bricks
-
A command line wizard. Once you get it running, it creates sleek applications.
RailsBricks enables you to cre...
Terminator setup for Procfile-based applications for more comfortable debugging
We use foreman
to start all necessary processes for an application, which are declared in a Procfile
. This is very convenient, but the outputs of all processes get merged together. Especially while debugging you might not want other processes to flood your screen with their log messages.
The following setup allows you to start Terminator in a split view with the Rails server running in the left pane and all remaining processes running via foreman in the right pane. It was heavily inspired by [this card](https://makandracards.com/makandr...
Organize large I18n dictionary files in Ruby on Rails
If you're suffering from a huge de.yml
or similiar file, cry no more. Rails lets you freely organize your dictionary files in config/locales
.
My organization works like this:
-
config/locales/rails.de.yml
modified Rails boilerplate -
config/locales/faker.de.yml
modified Faker boilerplate -
config/locales/models.de.yml
model names, attribute names, assignable_value labels - `config/locales/views.de.y...
Understanding SQL compatibility modes in MySQL and MariaDB
MySQL and MariaDB have an SQL mode setting which changes how MySQL behaves.
The SQL mode value is comprised of multiple flags like "STRICT_TRANS_TABLES, NO_ZERO_IN_DATE"
. Each flag activates or disables a particular behavior.
The default SQL mode varies widly between versions of MySQL and MariaDB. In general, more recent versions of MySQL and MariaDB have stricter settings than older versions, and MySQL has stricter settings than the more liberal MariaDB.
If your app explodes ...
Using the ActiveSupport::BroadcastLogger
The ActiveSupport::BroadcastLogger allows you to log to multiple sinks. You know this behavior from from the rails server
command, that both logs to standard out and the log/development.log
file.
Here is an example from the ActiveSupport::BroadcastLogger
API:
stdout_logger = ActiveSupport::Logger.new(STDOUT)
file_logger = ActiveSupport::Logger.new("development.log")
broadcast = ActiveSupport::BroadcastLogger.new(stdout_logger, file_logger)
broadcast.i...
How to: Upgrade CarrierWave to 3.x
While upgrading CarrierWave from version 0.11.x to 3.x, we encountered some very nasty fails. Below are the basic changes you need to perform and some behavior you may eventually run into when upgrading your application. This aims to save you some time understanding what happens under the hood to possibly discover problems faster as digging deeply into CarrierWave code is very fun...
Whitelists and blacklists
The following focuses on extension allowlisting, but it is the exact same thing for content type allowlisting with the `content_ty...
Upgrading from Capistrano 2 to 3
Capistrano 3 is a major rework of the framework and requires several adjustments to your deploy configuration files. The biggest change is that they moved away from their custom DSL and use Rake
instead. For connecting with and operating on the servers, they bring a new gem SSHKit
which does the heavy lifting. It's SSHKit's DSL that is used anywhere inside the Rake tasks. See #Resources at the bottom for examples.
Step 1: Upgrade guide
For migration from 2 to 3, follow this tutorial: [Capistrano 3 Upgrade Guide](https://semaphorec...
How to monitor Sidekiq: A working example
In order to have monitoring for Sidekiq (like queue sizes, last run of Sidekiq) your application should have a monitoring route which returns a json looking like this:
{
"sidekiq": {
"totals": {
"failed": 343938,
"processed": 117649167
},
"recent_history": {
"failed": {
"2016-11-06": 1,
"2016-11-07": 46,
"2016-11-08": 0,
"2016-11-09": 0,
"2016-11-10": 0
},
"processed": {
"2016-11-06": 230653,
"2016-11-07": 230701,
"2016-11-08"...
request_store: Per-request global storage for your Rails app
Ever needed to use a global variable in Rails? Ugh, that's the worst. If you need global state, you've probably reached for
Thread.current
.
When you're using Thread.current
, you must make sure you're cleaning up after yourself. Else, values stored in one request may be available to the next (depending on your server). request_store wipes all data when a request ends and makes per-request global storage a no-brainer. Internally, it's using Thread.current
with a Hash
in a simple middleware.
Example: Remembering all currently a...
How to open files from better_errors with RubyMine on Linux
I recently noticed that better_errors
allows you to to open files from within your favorite editor. However it was not so easy to get rubymine://
links to work on Gnome/Linux. Here is how it finally worked for me:
Step 1: Add a Desktop launcher
Add this file to ~/.local/share/applications/rubymine.desktop
:
[Desktop Entry]
Version=1.0
T...
Local development with SSL and Puma
Sometimes the need arises for SSL in local development. We have guides for different webservers, this one is for puma.
-
make sure mkcert is installed
-
create an SSL certificate for localhost with mkcert:
$ mkcert-v1.4.4-linux-amd64 localhost
Created a new local CA 💥
...
- use the certificate in the Puma config
config/puma.rb
:
localhost_key = "#{File.join('localhos...
Manage Linux services on the command line (Ubuntu)
Ubuntu 18.04 uses systemd
to manage services.
There are basically two commands for listing all services and manipulating the state of a certain service: service
and systemctl
:
-
service
manages System V init scripts -
systemctl
controls the state of the systemd system and service manager. It is backwards compatible to System V and includes the System V services
Therefore I prefer to use systemctl
.
See which services are there
>systemctl list-units -a --type=service
UNIT LOAD ...
Capistrano: Configure environment specific array attributes
Using Capistrano, we usually have some array configurations in the config/deploy.rb
file, like set :linked_files, %w[config/database.yml]
, so in this case we don't have to manage the database configuration individually on every server.
In a specific case, one of our projects supports sign in by SAML
, but not every deploy target has this feature activated. Here comes a nice handy Capistrano feature, which lets us modify the default configuration for individual env...
Reverse-proxying web applications with Apache 2.4+
Note: Making a reverse proxy with nginx is much more straightforward.
A reverse proxy is a "man in the middle" server that tunnels requests to another server. You can use for things like:
- Expose a local service that you cannot directly reach over the internet
- "Change" the domain or path of a web application by rewriting them on the fly
- Instantly change servers that respond to a name or ...
Some tips for upgrading Bootstrap from 3 to 4
Recently I made an upgrade from Bootstrap 3 to Bootstrap 4 in a bigger project. Here are some tips how to plan and perform such an upgrade. The effort will scale with the size of the project and its structure. If your stylesheets already follow strict rules, it may take less time to adapt them to the new version.
Preparation
There are several gems and libraries that works well with bootstrap or provide at least stylesheets/plugins to easily integrate the bootstrap theme. But very often they only work with specific version or are no long...
How Rails and MySQL are handling time zones
When working with times and dates in Rails applications, you need to deal with the following problem:
- In Rails,
Time
objects have a time zone. You can get the zone name by doingtime_object.zone
. - This zone is considered when doing time calculations, e.g. 10 AM CEST minus 8 AM UTC is zero.
- A datetime in MySQL does not have a zone. It just stores the literal string "2010-05-01 12:00:00".
- That means that Rails must make assumptions about timestamps loaded from and written to MySQL.
Rails has two completely different modes ...
How to fix parallel_tests with Redis on powerful machines
When you have a powerful machine with many CPU cores, you might run into an error like
ERR DB index is out of range (Redis::CommandError)
This is because Redis defaults to at most 16 databases (0 to 15) and running tests in parallel might exceed that (your tests might run on databases 1..n
or 2..(n+1)
).
You can increase that limit:
-
Get number of CPUs of your machine.
nproc --all
-
Open up Redis configuration file.
sudo vim /etc/redis/redis.conf
-
Find
databases
row and increase it, e.g. set to 32:
...
Version 5 of the Ruby Redis gem removes Redis.current
Redis.current
will be removed without replacement in redis-rb
5.0.
Version 4.6.0 adds deprecation warnings for Redis.current
and Redis.current=
:
`Redis.current=` is deprecated and will be removed in 5.0.
If your application still uses Redis.current
, you can only fix it by no longer using it. Here is how.
Redis.new when you need it
You can easily instantiate a Redis
client when you need it.
There is probably already a constant like REDIS_URL
that you use to configure Sidekiq or similar. So just use that one.
``...
Setup Sidekiq and Redis
If you want Sidekiq to be able to talk to Redis on staging and production servers, you need to add the following to your configuration:
# config/initializers/sidekiq.rb
require 'sidekiq'
Sidekiq.configure_client do |config|
config.redis = { url: REDIS_URL }
end
Sidekiq.configure_server do |config|
config.redis = { url: REDIS_URL }
end
The following step may be skipped for new Sidekiq 6+, since it isn't recommended anymore to use a global redis client.
# config/initializers/redis.rb
require 'redis'
require_relativ...
Rails: render a template that accepts a block by using the layout option of render
Let's say you have a form that you render a few times but you would like to customize your submit section each time. You can achieve this by rendering your form partial as layout and passing in a block. Your template or partial then serves as the surrounding layout of the block that you pass in. You can then yield back the form to the block and access the form in your block.
-# record/_form.haml
= form_for record do |form|
-# ...
.form-actions
yield(form)
In order to make your template record/_form.haml
accept a...
Fixing "identify: not authorized"
Ubuntu has decided to disable PDF processing because ImageMagick and the underlying Ghostscript had several security issues.
When your Ghostscript is up to date (and receiving updates regularly), you can safely reactivate PDF processing on your computer like this:
- Open
/etc/ImageMagick-6/policy.xml
(requires sudo)- For older versions of Ubuntu (or possibly ImageMagick), the path is
/etc/ImageMagick/policy.xml
- For older versions of Ubuntu (or possibly ImageMagick), the path is
- Remove/Comment lines after
<!-- disable ghostscript format types -->
If you need ...
Capistrano: creating a database dump if migrating
In Capistrano 3, your Capfile requires 'capistrano/rails/migrations'
, which brings two Capistrano tasks: deploy:migrate
and deploy:migrating
. The former checks whether migrations should be performed. If so, the latter is invoked, which performs the actual migrations.
Knowing this, it is easy to dump the db only if migrations will run. First, enable conditional migrations:
# config/deploy.rb
set :conditionally_migrate, true # Only attempt migration if db/migrate changed
Then hook up the dump task to deploy:migrating
:
Setting up Tomcat to use an existing OpenSSL certificate
This is for those who already own an SSL certificate (e.g. using it in the Apache HTTP Server) and need to feed it to a Tomcat application. The main issue is that you need to convert your OpenSSL certificate for Java to use it in its keystore.
Create a keystore
This is the place where the application usually gets its keys from. The keytool
is able to read certificate files but does not understand private key files which you need to use y...