How to checkout submodules in Gitlab CI

Accessing other repositories in Gitlab CI is not straight forward, since the access rights of the current pipeline might not be sufficient enough.

One approach is to use project access tokens and clone the repositories via HTTPS.

  • Create a project access token for all submodules you want to have access to with the setting read_repository
  • Add the secrets as environment variable to the main project you want to have access to submodules:
    • Protected false ...

NVM: How to automatically switch version when changing directories

The Node Version Manager allows installing multiple NodeJS versions and switching between them.
By default, it does not automatically switch versions when entering a directory that holds a .nvmrc file.

The project's readme document offers a bash function which calls nvm use after each cd. In fact, it replaces cd in your bash.

I did not want to do that, but instead use the $PROMPT_COMMAND feature. So here is my take on it.
Note that it is much shorter, it probably does a f...

Ruby: Fixing strings with invalid encoding and converting to UTF-8

When dealing with external data sources, you may have to deal with improperly encoded strings.
While you should prefer deciding on a single encoding with the data-providing party, you can not always force that on external sources.
It gets worse when you receive data with encoding declaration that does not reliably fit the accompanying string bytes.

Here is a Ruby class that helps converting such strings to a proper encoding.
Note that it tries several approaches of changing the encoding. **This is not a silver bullet and may or may not work...

When does Webpacker compile?

Webpack builds can take a long time, so we only want to compile when needed.

This card shows what will cause Webpacker (the Rails/Webpack integration) to compile your assets.

When you run a dev server

While development it is recommended to boot a webpack dev server using bin/webpack-dev-server.

The dev server compiles once when booted. When you access your page on localhost before the initial compilation, the page may load without assets.

The ...

Bundler: Packaging gems into the git repository (offline installation)

Installing gems on a server that has no access to the internet (especially rubygems.org) requires to bundle the gems into the repository itself. This requires to adjust the bundle config in the repository.

  1. Execute the following commands to configure bundler:
bundle config set --local path vendor
bundle config set --local disable_shared_gems true

Note

For Bundler < 2 you have to omit the "set": bundle config --local name value.
See here: [https://bundler.io/v1.17/man/bundle-config.1.html](https://bundler.io/v1.17/man...

How to push to Git without running CI on GitLab CI, GitHub Actions, or Travis CI

If a project ist configured to spawn CI runners for tests or deployment when pushing to the Repo, a habit of pushing WIP commits regularly may conflict with that.
Here are two solutions that allow you to keep pushing whenever you feel like it.

Special commit message

To skip a CI run, simply add [ci skip] or [skip ci] to your commit message. Example:

git commit -m "wip authentication [ci skip]"

Git push options (GitLab)

In addition to that, GitLab CI supports Git push options. Instead of changing your commit message, ...

Delivering Carrierwave attachments to authorized users only

Preparation

To attach files to your records, you will need a new database column representing the filename of the file. To do this, add a new migration (rails g migration <name>) with the following content:

class AddAttachmentToNotes < ActiveRecord::Migration[6.0]
  def change
    add_column :notes, :attachment, :string
  end
end

Don't forget to rename the class and change the column details to fit your purpose. Run it.

1) Deliver attachments through Rails

The first way is to store your Carrierwave attachments not ...

Capybara: Preventing headless Chrome from freezing your test suite

We prefer to run our end-to-end tests with headless Chrome. While it's a very stable solution overall, we sometimes see the headless Chrome process freeze (or the Capybara driver losing connection, we're not sure).

The effect is that your test suite suddenly stops progressing without an error. You will eventually see an error after a long timeout but until then it will seem that your suite is frozen. If you're also using [capybara-screenshot](https:/...

Capybara: Preventing server errors from failing your test

When your Rails application server raises error, Capybara will fail your test when it clears the session after the last step. The effect is a test that passes all steps, but fails anyway.

Capybara's behavior will help you to detect and fix errors in your application code. However, sometimes your application will explode with an error outside your control. Two examples:

  • A JavaScript library references a source map in its build, but forgets to package the source map
  • CarrierWave cleans up an upload or cache file after the record was delet...

RSpec: How to test the content of a flash message in a request spec

The ActionDispatch module of Rails gives you the helper method flash to access the flash messages in a response.

describe PostsController, type: :request do

  describe 'update' do

    it 'shows a success message on update' do
      post_record = create(:post)

      put "/posts/#{post_record.id}"

      # Same as @request.flash[:alert]
      expect(flash[:alert]).to eq('Post updated successfully.')
    end

  end


end

Webmock < 3.12.1 cannot handle IPv6 addresses correctly

We had the issue, that a VCR spec failed, after updating CarrierWave from version 0.11.0 to 1.3.2.
In this version, CarrierWave uses the gem SsrfFilter, which retrieves the IP addresses for the given hostname and replaces the hostname in the requested url with one of them.

It works with IPv4 addresses, but not with IPv6 addresses, because WebMock cannot handle those correctly:

uri = "#{protocol}://...

Making ZSH the default shell on Ubuntu 20.04

ZSH is an alternative command line shell that includes some features like spelling correction, cd automation, better theme, and plugin support. You can replace Bash with ZSH like following:

sudo apt-get install zsh

Setting ZSH as default login shell

sudo usermod -s /usr/bin/zsh $(whoami)

Opening a new terminal window will show you a dialog where you can configure your initial ZSH config (Option 2 recommended).

Afterwards you can install the plugin manager Oh-My-ZSH and select a prop...

CarrierWave: Default Configuration and Suggested Changes

CarrierWave comes with a set of default configuration options which make sense in most cases. However, you should review these defaults and adjust for your project wherever necessary.

You will also find suggestions on what to change below.

Understanding the default configuration

Here is the current default config for version 2:

config.permissions = 0644
config.directory_permissions = 0755
config.storage_engines = {
  :f...

Running old ImageMagick versions in a Docker container

If your project depends on an old version of ImageMagick that you can no longer install in your system, you can choose the run an old ImageMagick in a Docker container.

Dockerized ImageMagick commands will only work with absolute path arguments. You need to boot a corresponding docker container once before using it.

Setting up Docker

If you haven't installed Docker yet, use our guide or the [official instructions](https://docs.docker.com/get-started/...

Parallelize Development Using Git Worktrees

You can use git worktree to manage multiple working trees attached to the same repository. But why should I use git worktree?

You can use more than one working tree to ...

... run tests while working on another branch
... compare multiple versions
... work on a different branch without disturbing your current branch

Creating a new working tree is as simple as creating a new branch. You only need to execute git worktree add <path> <branch>. When you are done, you can remove the working tree with git worktree remove <Worktree>...

Regular Expressions: Quantifier modes

When you repeat a subpattern with a *, + or {...} operator, you may choose between greedy, lazy and possessive modes.

Switching modes may affect the result and performance of your regular expressions. In the worst case, an ill-suited mode may make your regular expression so slow that it can DoS your application (Examples are the ActiveRecord's PostgreSQL CVE-2021-22880 or the [Cloudflare outage 2019](https://makandracards.com/makandra/77515-regular-expressions-excessive-backtracking...

Linux: How to make a terminal window title reflect the current path

By default, your terminal emulator (Gnome Terminal, Terminator, etc.) sets some kind of window title to reflect the shell type you are running (e.g. /bin/bash).
This is most often not too helpful, but you can change that from your shell.

To set a specific title, print an escape sequence like this:

echo -en "\033]0;Hello\a"

You can easily include the current path:

echo -en "\033]0;$(pwd)\a"

Or, to replace your home directory's part with a tilde:

echo -en "\033]0;$(pwd | sed -e "s;^$HOME;~;")\a"

Or,...

DNS debug tools

There are several tools for DNS debugging which offer you more or less information. Most of the time the more simple ones, like host oder nslookup will be sufficient.

host

simple DNS lookup utility.

>host heise.de
heise.de has address 193.99.144.80
heise.de has IPv6 address 2a02:2e0:3fe:1001:302::
heise.de mail is handled by 10 relay.heise.de.

nslookup

query Internet domain name servers. Nslookup has two modes: interactive and non-interactive.

>nslookup heise.de
Server:		146.254.160.30
Address:	146.254.160.3...

Git: Parsing large diffs as a human

I just finished migrating a project from the Asset Pipeline to Webpacker, this is what my diff to master looks like:

5.825 files changed, 44.805 insertions(+), 529.948 deletions(-)
warning: inexact rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 5134 and retry the command.

There is no way me or my peer reviewer is able to parse 500k+ lines of code. Fortunately, git has ...

Transporting blank values in URL queries

URLs can transport key/value pairs ("parameters") using this syntax:

/path?foo=bar

If the value is blank, mind these subtle differences:

URL Meaning
/path?foo= Parameters have a key foo. Its value is an empty string.
/path?foo Parameters have a key foo. Its value is null.
/path Parameters have no key foo.

How to build a fully custom TinyMCE 5 dialog

TinyMCE is a WYSIWYG editor which is quite customizable.


  1. Add a custom button to the tinyMCE toolbar and tell tinyMCE to open a dialog with the route to your dialog's view.
tinymce.init({
  // ...
  toolbar: 'myCustomButton',
  setup: function(editor) {
      editor.ui.registry.addButton('myCustom Button', {
        ...

Configuring Webpacker deployments with Capistrano

When deploying a Rails application that is using Webpacker and Capistrano, there are a few configuration tweaks that optimize the experience.

Using capistrano-rails

capistrano-rails is a Gem that adds Rails specifics to Capistrano, i.e. support for Bundler, assets, and migrations. While it is designed for Asset Pipeline (Sprockets) assets, it can easily be configured for Webpacker. This brings these features to the Webpacker world:

  • Automatic removal of expired assets
  • Manifest backups