Capybara: Running tests with headless Chrome

Headless Chrome is a way to run the Chrome browser without a visible window.

Configuring Capybara

Configure the Capybara driver like this:

Capybara.register_driver :selenium do |app|
  options = Selenium::WebDriver::Chrome::Options.new
  options.add_argument('--disable-infobars')
  options.add_emulation(device_metrics: { width: 1280, height: 960, touch: false })
  
  unless ENV.key?('NO_HEADLESS')
    options.add_argument('--headless')
    o...

SSHKit 1.9.0 failure for Capistrano deploy

SSHKit 1.9.0 might fail with the following error, when trying to deploy a Rail application. Upgrading the gem to version 1.21.0 fixed the issue.

Traceback (most recent call last):
	17: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
	16: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9.0/lib/sshkit/backends/abstract.rb:29:in `run'
	15: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9....

Whenever requires you to set the application attribute in the Capistrano config

Whenever requires you to set the application attribute in your Capistrano configuration. Otherwise your cronjobs are created multiple times.

Example entry in config/deploy.rb:

set :application, 'some-app' # allows "set :whenever_identifier, ->{ "#{fetch(:application)}_#{fetch(:stage)}" }" to work as expected

Good

Then the crontab -l output will look like this:

# Begin Whenever generated tasks for: som...

Missing certificates for rubygems and bundler in Ruby 1.8.7

Using Ruby 1.8.7 you will not be able to use the maximum versions Rubygems 1.8.30 and Bundler 1.17.3 with https://rubygems.org/ anymore. This is a result of a server certificate on December 5th, 2020. The resulting errors will look like following:

  • TypeError: can't modify frozen object
  • Could not verify the SSL certificate for https://rubygems.org/*
  • Bundler::Fetcher::CertificateFailureError: Could not verify the SSL certificate for https://index.rubygems.org/versions.
  • `Error fetching data: hostname was not m...

Configuring Webpacker deployments with Capistrano

When deploying a Rails application that is using Webpacker and Capistrano, there are a few configuration tweaks that optimize the experience.

Using capistrano-rails

capistrano-rails is a Gem that adds Rails specifics to Capistrano, i.e. support for Bundler, assets, and migrations. While it is designed for Asset Pipeline (Sprockets) assets, it can easily be configured for Webpacker. This brings these features to the Webpacker world:

  • Automatic removal of expired assets
  • Manifest backups

Show/Hide Rubocop marking in RubyMine

If you have installed Rubocop in your project, RubyMine can show you Rubocop violations immediately in your editor. You probably already know this feature.

Example

Image

Enable/Disable marking

If your RubyMine does not show you any violations, although there are some, you may have to enable the setting first.

To do so, open Navigate -> Search Everywhere -> Actions (Or use the shortcut CTRL + SHIFT + A) and type in "rubocop", then you should see some...

How to configure file watchers in RubyMine

Installation

You need to install the official plugin, it is not bundled with RubyMine by default.

Example: Setup a watcher to verify rubocop integrity

First, open Settings -> Tools -> File Watchers. Then, configure rubocop to check every change to the VCS:

Image

Note that the "program" argument must be part of your $PATH. I worked around this constraint by using b as a shim for bundle exec.

Resources

  • [File watchers documen...

Migrate gem tests from Travis CI to Github Actions with gemika

We currently test most of our gems on Travis CI, but want to migrate those tests to Github Actions. This is a step-by-step guide on how to do this.

Note that this guide requires the gem to use gemika.

  1. Go to a new "ci" branch:
    git checkout -b ci
    
  2. Update gemika to version >= 0.5.0 in all your Gemfiles.
  3. Have gemika generate a Github Actions workflow definition by running
    mkdir -p .github/workflows; bundle exec rake gemika:generate_github_actions_workflow > .github/workf...
    

Installing old versions of mysql2 on Ubuntu 20.04+

On some of our older projects, we use the mysql2 gem. Unfortunately, versions 0.2.x (required for Rails 2.3) and versions 0.3.x (required for Rails 3.2) can no longer be installed on Ubuntu 20.04. Trying this either leads to errors when compiling the native extension, or a segfaults when using it.

For Rails 4.2, mysql2 version 0.4.10 seems to work okay. If you still have issues, upgrade to 0.5.x, which should be compatible.

To install it, you have to switch the mysql2 gem to our fork. In your Gemfile, ...

Rails: How to restore a postgres dump from the past

It sometimes happen that a database dump, that would want to insert into your development database, does not match the current schema of the database. This often happens when you have an old dump, but your current setup is up to date with the the master.

Hint: In most cases it is sufficient to delete and recreate the local database in order to import the dump. If any problems occur, proceed as follows:

1. Figure out the original migration status of the dumpfile

  • Convert your dump to plaintext: `pg_restore -f some.dump > some.dump....

How to implement simple queue limiting/throttling for Sidekiq

The sidekiq-rate-limiter gem allows rate-limiting Sidekiq jobs and works like a charm. However, it needs to be integrated on a per-worker basis.

If you want to limit a whole queue instead, and if your requirements are simple enough, you can do it via a Sidekiq middleware yourself.

Here is an example that limits concurrency of the "mailers" queue to 1. It uses a database mutex via the [with_advisory_lock](https://github.com/ClosureTree/wit...

VCR and the webdrivers gem

If you're using the webdrivers gem and VCR together, depending on your configuration, VCR will yell at you regulary.
The webdrivers gem tries to update your webdrivers on your local machine. To do so, it checks the internet for newer versions, firing an HTTP-request to e.g. https://chromedriver.storage.googleapis.com

You can "fix" this in multiple ways:

  1. Update your drivers on your machine with
    RAILS_ENV=test rake webdrivers:chromedriver:update

  2. Ignore the driver update-URL in your ...

Pagy

Pagy is a gem for pagination.
They make some bold claims:

Pagy is the ultimate pagination gem that outperforms the others in each and every benchmark and comparison.

Maybe this is worth trying out.

Workflow: How to use a key management service to encrypt passwords in the database

This is an extract from the linked article. It shows an approach on how to implement encrypted passwords with the AWS Key Management Service (KMS).

For most applications it's enough to use a hashed password with a salt (e.g. the gem devise defaults to this).

Upon password creation

  1. Generate hash as hash of password + salt.

  2. Encrypt the hash with a public key from KMS (you can store the public key in your server code).

  3. In your database sto...

How to fix: Rails query logs always show lib/active_record/log_subscriber.rb as source

Rails 5.2+ supports "verbose query logs" where it shows the source of a query in the application log.
Normally, it looks like this:

  User Load (0.5ms)  SELECT "users".* FROM "users" WHERE ...
  ↳ app/controllers/users_controller.rb:42:in `load_users'

However, you may encounter ActiveRecord's LogSubscriber as the source for all/most queries which is not helpful at all:

  User Load (0.5ms)  SELECT "users".* FROM "users" WHERE ...
  ↳ activerecord (6.0.3.3) lib/active_record/log_subscriber.rb:100:in `debug'

While th...

Controlling issue grouping in Sentry

When you use Sentry to monitor exceptions, an important feature is Sentry's error grouping mechanism. It will aggregate similar error "events" into one issue, so you can track and monitor it more easily. Grouping is especially important when you try to silence certain errors.

It is worth understanding how Sentry's grouping mechanism works.

The default grouping mechanism

The exact algorithm has changed over time, and Sentry will keep using the algorithm t...

Parallel cucumber: How to pass in cucumber arguments

Here is an example with the --tags option. You need to wrap them inside --cucumber-options option of parallel_cucumber.

DISPLAY=:17 bundle exec parallel_cucumber --cucumber-options '--tags @solo' features

See more details in the docs.

PostgreSQL: How to use with_advisory_lock to prevent race conditions

If you want to prevent that two processes run some code at the same time you can use the gem with_advisory_lock.

What happens

  1. The thread will wait indefinitely until the lock is acquired.
  2. While inside the block, you will exclusively own the advisory lock.
  3. The lock will be released after your block ends, even if an exception is raised in the block.

This is usually required if there is no suitable database row to lock on.

Example

You want to generate a...

Vortrag: Content Security Policy: Eine Einführung

Grundidee

CSP hat zum Ziel einen Browser-seitigen Mechanismus zu schaffen um einige Angriffe auf Webseiten zu verhindern, hauptsächlich XSS-Angriffe.

Einschub: Was ist XSS?

XSS = Cross Site Scripting. Passiert wenn ein User ungefiltertes HTML in die Webseite einfügen kann.

<div class="comment">
  Danke für den interessanten Beitrag! <script>alert('you have been hacked')</script>
</div>

Rails löst das Problem weitgehend, aber

  • Programmierfehler weiter möglich
  • manchmal Sicherheitslücken in Gems oder Rails

Lösungsid...

Introducing GoodJob 1.0, a new Postgres-based, multithreaded, ActiveJob backend for Ruby on Rails

GoodJob is a new background worker gem. It's compatible with ActiveJob.

We're huge fans of Sidekiq for its stability and features. One advantage of GoodJob over Sidekiq is that GoodJob doesn't require Redis. So in cases where you don't have Redis or don't want to pay for a Redis HA quorum node, this might be an alternative worth checking out.

Fixing wall of warnings: already initialized constant Etc::PC_SYMLINK_MAX

These warnings are printed when the etc Gem is installed, while etc is also included in Ruby. Fix with:

gem uninstall etc

Automatically validating dependency licenses with License Finder

"Open-source software (OSS) is great. Anyone can use virtually any open-source code in their projects."

Well, it depends. Licenses can make things difficult, especially when you are developing closed-source software. Since some OSS licenses even require the employing application to be open-sourced as well (looking at you, GPL), you cannot use such software in a closed-source project.

To be sure on this, we have developed a project-level integration of Pivotal's excellent [license_finder](https:/...

PostgreSQL: Difference between text and varchar columns

TL;DR PostgreSQL handles Rails 4+ text and string columns the same. Some libraries may still reflect on the column type and e.g. render differently sized text fields.


PostgreSQL offers three character types for your columns:

  • character varying(n) (also called varchar or just string): Contents are limited to n characters, smaller contents are allowed.
  • character(n): All contents are padded with spaces to allocate exactly n characters.
  • text: There is no upper or lower character limit (except for the absolute...

How to use Active Job to decouple your background processing from a gem

In a web application you sometimes have tasks that can not be processed during a request but need to go to the background.
There are several gems that help to you do that, like Sidekiq or Resque.

With newer Rails you can also use ActiveJob as interface for a background processing library. See here for a list of supported queueing adapters.
For ...