Fixing "identify: not authorized"

Ubuntu has decided to disable PDF processing because ImageMagick and the underlying Ghostscript had several security issues.

When your Ghostscript is up to date (and receiving updates regularly), you can safely reactivate PDF processing on your computer like this:

  1. Open /etc/ImageMagick-6/policy.xml (requires sudo)
    • For older versions of Ubuntu (or possibly ImageMagick), the path is /etc/ImageMagick/policy.xml
  2. Remove/Comment lines after <!-- disable ghostscript format types -->

If you need ...

Setting up Tomcat to use an existing OpenSSL certificate

This is for those who already own an SSL certificate (e.g. using it in the Apache HTTP Server) and need to feed it to a Tomcat application. The main issue is that you need to convert your OpenSSL certificate for Java to use it in its keystore.

Create a keystore

This is the place where the application usually gets its keys from. The keytool is able to read certificate files but does not understand private key files which you need to use y...

Chromedriver: Connect local chromedriver with docker

Debugging your integration tests, that run a headless Chrome inside a docker image, is tricky.

In many cases you can connect your Chrome to a remote docker container like docker-selenium, which should be the preferred way when you try to inspect a page within your integration test.

Otherwise you might be able to start your docker container with --net=host and access your local chromedriver in the host address space host.docker.internal.

If both options above don't work for you here is a...

Local development with SSL and Puma

Sometimes the need arises for SSL in local development. We have guides for different webservers, this one is for puma.

  1. make sure mkcert is installed

  2. create an SSL certificate for localhost with mkcert:

$ mkcert-v1.4.4-linux-amd64 localhost
Created a new local CA 💥
...
  1. use the certificate in the Puma config config/puma.rb:
localhost_key = "#{File.join('localhos...

MySQL: Disable query cache for database profiling

If you want to see how long your database queries actually take, you need to disable MySQL's query cache. This can be done globally by logging into a database console, run

SET GLOBAL query_cache_type=OFF;

and restart your rails server.

You can also disable the cache on a per query basis by saying

SELECT SQL_NO_CACHE * FROM ...

You also probably want to disable Rails internal (per-request) cache. For this, wrap your code with a call to ActiveRecord::Base.uncached. For example, as an around_filter:

d...

esbuild: Make your Rails application show build errors

Building application assets with esbuild is the new way to do it, and it's great, especially in combination with Sprockets (or Propshaft on Rails 7).
You might be missing some convenience features, though.

Here we cover one specific issue:
Once you have started your development Rails server and esbuild with the --watch option (if you used jsbundling-rails to set up, you probably use bin/dev), esbuild will recompile your assets upon change, but build errors will only be printed to the terminal. Your application won't complain about them ...

Missing certificates for rubygems and bundler in Ruby 1.8.7

Using Ruby 1.8.7 you will not be able to use the maximum versions Rubygems 1.8.30 and Bundler 1.17.3 with https://rubygems.org/ anymore. This is a result of a server certificate on December 5th, 2020. The resulting errors will look like following:

  • TypeError: can't modify frozen object
  • Could not verify the SSL certificate for https://rubygems.org/*
  • Bundler::Fetcher::CertificateFailureError: Could not verify the SSL certificate for https://index.rubygems.org/versions.
  • `Error fetching data: hostname was not m...

Heads up: Deployment with newly generated SSH key (using ED25519) might fail

If you use a newer SSH key generated with the ED25519 algorithm instead of RSA (see Create a new SSH key pair), the deployment with Capistrano may fail with the following message:

The deploy has failed with an error: unsupported key type `ssh-ed25519'
net-ssh requires the following gems for ed25519 support:
 * ed25519 (>= 1.2, < 2.0)
 * bcrypt_pbkdf (>= 1.0, < 2.0)
See https://github.com/net-ssh/net-ssh/issues/565 for more information
Gem::LoadError : "ed25519 i...

JavaScript: Sharing content with the native share dialog

Mobile Chrome and Safari support the "web share API" which allow you to use the native share functionality of an Android or iOS phone. Some desktop OSs like Windows or MacOS also support native share dialogs. See Can I Use for a detailed support matrix.

When clicking a share button using this API, the browser will automatically show all installed applications that support content sharing, such as Whatsapp, Facebook, Twitter, e-mail etc.

The API is extremely simple to use:

if ...

Change / Update SSL certificate for Amazon Elastic Load Balancer

There is a new card about how to do this with the new AWS Command Line Interface


At first you need the IAM Cli Tools.
-------------------------------------------------------------------------------------------------------------...

jQuery and cross domain AJAX requests

When making cross-domain AJAX requests with jQuery (using CORS or xdomain or similar), you will run into issues with HTTP headers:

  • jQuery will not set the X-Requested-With header. On your server, requests will not look like AJAX requests (request.xhr? will be false).
  • jquery-ujs will not set CSRF headers.

This is by design and improves secu...

Better HTML forms: use type, inputmode, enterkeyhint and autocomplete

Web forms can be made much more usable with a few HTML attributes. Short summary:

  • type: Tells browsers about the input data type. Mobile browsers will select a virtual keyboard based on this value. Some browsers will add simple validation, e.g. for type email.
  • inputmode: Direct hint about the virtual keyboard to use. Inferred from type, but can be very handy when ...

Carrierwave: How to migrate to another folder structure

A flat folder structure can be cool if you have only a few folders but can be painful for huge amounts. We recently had this issue in a project with more than 100.000 attachments, where we used a structure like this /attachments/123456789/file.pdf.

Even the ls command lasted several minutes to show us the content of the attachments folder.

So we decided to use a more hierarchical structure with a limited maximum of folder per layer. Here are a few tips how to migrate your files to their new...

A saner alternative to SimpleForm's :grouped_select input type

SimpleForm is a great approach to simplifying your forms, and it comes with lots of well-defined input types. However, the :grouped_select type seems to be overly complicated for most use cases.

Example

Consider this example, from the documentation:

form.input :country_id, collection: @continents,
  as: :grouped_select, group_method: :countries

While that looks easy enough at a first glance, look closer. The example passes @continents for a country_id.\
SimpleForm actua...

Debug flaky tests with an Unpoly observeDelay

The problem

Unpoly's [up-observe], [up-autosubmit] and [up-validate] as well as their programmatic variants up.observe() and up.autosubmit() are a nightmare for integration tests.

Tests are usually much faster than the configured up.form.config.observeDelay. Therefore, it may happen that you already entered something into the next field before unpoly updates that field with a server response, discarding your changes.

The steps I wait for active ajax requests to complete (if configured) and capybara-lockstep can catch some ...

Read your mail in networks that forbid e-mail traffic

If you are connected with a network that forbids e-mail traffic but allows SSH, you can tunnel your e-mail connection through a trusted, intermediary server:

 sudo ssh -i /home/local-user/.ssh/local-user.key -L 143:mail-server.tld:143 remote-user@trusted-server.tld

Explanation for the command above:

| sudo | Secure ports may only be forwarded by root |
| ssh -i /home/local-user/.ssh/local-user.key | Private half of your SSH key for the trusted server |
| -L 143:mail-server.tld:143 | Forward connections to port 143 on your mail s...

Carrierwave: Using a nested directory structure for file system performance

When storing files for lots of records in the server's file system, Carrierwave's default store_dir approach may cause issues, because some directories will hold too many entries.

The default storage directory from the Carrierwave templates looks like so:

class ExampleUploader < CarrierWave::Uploader::Base
  def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end
end

If you store files for 500k records, that store_dir's parent directory will have 500k sub-directories which will cause some...

Debugging Capistrano

Capistrano 3 has a doctor task that will print information about

  • Environment: Ruby, Rubygems and Bundler versions
  • List of Capistrano gems and whether an update is available
  • All config variables and their values
  • Capistrano server config
$ bundle exec cap staging doctor

How to fix "unknown role" errors in Capistrano recipes

When you have a complex recipe setup with multistage deployment you may run into this error:

`role_list_from': unknown role `something' (ArgumentError)

Consider this task definition:

namespace :foo do
  task :bar, :roles => :something do
    # do crazy stuff
  end
end

Whenever we call foo.bar in our recipe, Capistrano will fail if you deploy to a stage where none of the servers has the role the error complains about, "something" in this case.

However, you can [hack around it](http://groups.google.com/group/ca...

Workflow: How to use a key management service to encrypt passwords in the database

This is an extract from the linked article. It shows an approach on how to implement encrypted passwords with the AWS Key Management Service (KMS).

For most applications it's enough to use a hashed password with a salt (e.g. the gem devise defaults to this).

Upon password creation

  1. Generate hash as hash of password + salt.

  2. Encrypt the hash with a public key from KMS (you can store the public key in your server code).

  3. In your database sto...

Capistrano 3: How to deploy when a firewall blocks your git repo

Sometimes, through some firewall or proxy misconfiguration, you might have to deploy to a server that cannot access the git repository.

Solution 1: HTTP Proxy (this is the preferred fix)

SSH can be tunneled over an HTTP Proxy. For example, when the repo is on github, use this:

  1. Install socat

  2. Add a ~/.ssh/config on the target server(s) with permission 0600 and this content:

    Host github.com ssh.github.com
      User git
      Hostname ssh.github.com
      Port 443
      ProxyCommand socat - PROXY:<your proxyhost>:%h:%p,...
    

makandra tech survey - results

These are the results of the "personal tech stack survey". I've included only the most popular mentions, maybe it can help you find one or two useful tools for your own usage.

Desktop environment

pie title Desktop environment
    "Gnome" : 16
    "i3": 2
    "sway": 2
    "awesome": 1
    "bspwm": 1
    "mate": 1
    "xfce": 1

Gnome dominates (unsuprising, it's the Ubuntu default), but quite a few people use tiling window managers, most popular i3 and the mostly i3-compatible [sway](https://swaywm....

Cronjobs: "Craken" is dead, long live "Whenever"

Our old solution for cronjobs, the "craken" plugin, is no longer maintained and does not work on Rails 3.2+.

We will instead use the whenever gem.

"Whenever" works just like "craken", by putting your rake tasks into the server's cron table. Everything seems to work just like we need it.

Installation for new projects

  1. Add "whenever" to your Gemfile:

     group :deploy do
       gem 'whenever', require: false
     end
    
  2. Add it to your config/deploy.rb:
    ...

Loading dumps via SSH, unpacking and sourcing them, all with a progress bar

Here is a hacky way to load dumps directly from the source server, without fully copying them over and extracting them first.

It may break horribly for you. This is the dark side of the force.

  1. Install pipe viewer, if you don't have it already: sudo apt-get install pv
  2. Know the location of the dump file on the remote server. We'll use /mnt/dumps/my_project.dump.bz2 in the example below.
  3. Find out the size of the (bzipped) file in by...