How to checkout submodules in Gitlab CI

Accessing other repositories in Gitlab CI is not straight forward, since the access rights of the current pipeline might not be sufficient enough.

One approach is to use project access tokens and clone the repositories via HTTPS.

  • Create a project access token for all submodules you want to have access to with the setting read_repository
  • Add the secrets as environment variable to the main project you want to have access to submodules:
    • Protected false ...

Upgrading Capybara with deprecated Integer selectors

Capybara added a deprecation warning in version 3.35.3 (version from 2019) that shows up if your selector is not of type String or Symbol.

Example:

click_link(10) # bad
click_link("10") # good

You might encounter this error e.g. in a pagination step or similar where you want to click on numbers. To figure out where this deprecation warning comes from try to run the tests with a step output.

bundle exec parallel_cucumber --test-options "--format=pretty" feature

The deprecation message looks like following:

Locator In...

Chromedriver: Disabling the w3c option might break your integration tests with Chrome 91

We recently noticed issues with Chrome 75+ when having the w3c option enabled within the Selenium webdriver. It looks like recent Selenium versions don't have any issues with the w3c interface anymore. And starting with Chrome 91 this fix might cause unexpected issues, so you should try to enabled this option again or just remove the following line from you configuration:

options.add_option('w3c', false)

Background: Setting the w3c option t...

Heads up: counting may be slow in PostgreSQL

The linked article points out that COUNT queries might be unexpectedly slow in PostgreSQL.

If you just need to know "are there any records" use any?. This uses SELECT 1 AS one FROM ... LIMIT 1 under the hood.
If you just need to know "are there no records" use empty? or none?. This uses SELECT 1 AS one FROM ... LIMIT 1 under the hood.

In short: Replace foo.count > 0 with foo.any? or foo.count == 0 with foo.none?

If you require quick counts and can tolerate some level of imprecision, consider exploring the ...

Pre-releasing an npm package

You can publish pre-release versions of an npm package.

Naming convention for pre-release versions

An npm package must use Semantic Versioning's naming convention for its version. In Semantic Versioning, the version number and pre-release identifier (like rc1) must be separated by a dash, like this:

  • 1.0.0-rc1
  • 2.3.0-alpha2
  • 3.0.0-beta3

Publishing to a pre-release tag

npm packages have multiple "current" releases, identified by "tags". The default tag is latest. It is expected to contain the la...

Pre-releasing a Ruby gem

When a Ruby version gem has a letter in its version number, it is considered a pre-release:

  • 1.0.0.rc1
  • 2.3.0.alpha2
  • 3.0.0.beta3
  • 4.0.0.pre.rc2

Even if a pre-release gem has the highest version number, it is never installed unless the user explictily requested the version:

gem install foobar --version="=2.3.0.alpha2"

Also bundle update will never update a stable version to a pre-release version unless the user explicitly requests it in the Gemfile:

gem 'foobar', '=2.3.0.alpha2'

A note on Semanti...

Webpacker: Loading code on demand

Sometimes you want to load code on demand. For instance, when a a large library is only used on a single screen, this library should only be loaded for that screen. It should not blow up the bundle size for all screens.

You can load code on demand by using import() as a function (with parentheses) instead of using it as a keyword import (without parentheses). The import() function returns a promise with the exported variables:

let exports = await import('large-library')

console.log("A named export is ", exports.exportedName)
c...

Limiting GitLab CI runner to specific branches or events

Use rules to include or exclude jobs in pipelines.

Rules are evaluated in order until the first match. When a match is found, the job is either included or excluded from the pipeline, depending on the configuration. The job can also have certain attributes added to it.

rules replaces only/except and they can’t be used together in the same job. If you configure one job to use both keywords, the linter returns a key may not be used with rules error.

GitLab 12.3 introduced rules. You can use them in your .gitlab-ci.yml in your proj...

Bundler: Packaging gems into the git repository (offline installation)

Installing gems on a server that has no access to the internet (especially rubygems.org) requires to bundle the gems into the repository itself. This requires to adjust the bundle config in the repository.

  1. Execute the following commands to configure bundler:
bundle config set --local path vendor
bundle config set --local disable_shared_gems true

Note

For Bundler < 2 you have to omit the "set": bundle config --local name value.
See here: [https://bundler.io/v1.17/man/bundle-config.1.html](https://bundler.io/v1.17/man...

Capistrano: Deployment issue undefined method `[]' for nil:NilClass

In newer passenger versions the output of passenger -v has changed. capistrano-passenger tries to parse the version and now causes the error undefined method '[]' for nil:NilClass. To fix this you only need to upgrade the capistrano-passenger gem.

Therefore run bundle update capistrano-passenger --conservative.

The version change of passenger from 6.0.7 to 6.0.8 has triggered this problem. This is fixed in capistrano-passenger >= 0.2.1.

Changes to positional and keyword args in Ruby 3.0

Ruby 3.0 introduced a breaking change in how it treats keyword arguments.

There is an excellent blog post on the official Ruby blog going into the details. You should probably read that, but here is a slightly abbreviated version:

What changed

When the last argument of a method call is a Hash, Ruby < 3 automatically converted it to to Keyword Arguments. If you call a method in Ruby >= 3 that accepts keyword arguments, eithe...

makandra/capybara-lockstep

capybara-lockstep can help you with flaky end-to-end tests:

This Ruby gem synchronizes Capybara commands with client-side JavaScript and AJAX requests. This greatly improves the stability of a full-stack integration test suite, even if that suite has timing issues.

Parsing JSON with edge cases

The linked article shows that there are unclear parts in the JSON specification and that different parsers treat them differently (which could lead to security vulnerabilities in certain cases).

I was curious what Ruby does (Ruby 2.6.6 with gem json 2.3.0, implementing RFC 7159):

Duplicate Keys

irb(main):001:0> require 'json'
=> true
irb(main):002:0> JSON.parse('{"qty": 1, "qty": -1}')
=> {"qty"=>-1}

Character Collision

irb(main):009:0> JSON.parse('{"qty": 1, "qty\ud800": -1}')
JSON::ParserError (487: incomplete sur...

Webmock < 3.12.1 cannot handle IPv6 addresses correctly

We had the issue, that a VCR spec failed, after updating CarrierWave from version 0.11.0 to 1.3.2.
In this version, CarrierWave uses the gem SsrfFilter, which retrieves the IP addresses for the given hostname and replaces the hostname in the requested url with one of them.

It works with IPv4 addresses, but not with IPv6 addresses, because WebMock cannot handle those correctly:

uri = "#{protocol}://...

Too many parallel test processes may amplify flaky tests

By default parallel_tests will spawn as many test processes as you have CPUs. If you have issues with flaky tests, reducing the number of parallel processes may help.

Important

Flaky test suites can and should be fixed. This card is only relevant if you need to run a flaky test suite that you cannot fix for some reason. If you have no issues...

How to fix: WrongScopeError when using rspec_rails with Rails 6.1

tl;dr: Upgrade the gem to at least 4.0.1

When you use rspec_rails in a version < 4 with Rails 6.1 you may encounter an error like this:

Failure/Error:
  raise WrongScopeError,
    "`#{name}` is not available from within an example (e.g. an " \
    "`it` block) or from constructs that run in the scope of an " \
    "example (e.g. `before`, `let`, etc). It is only available " \
    "on an example group (e.g. a `describe` or `context` block)."
    `name` is not available from within an example (e.g. an `it` block) or from constructs that...

How to list updateable dependencies with Bundler and Yarn

Bundler

bundle outdated [--filter-major|--filter-minor|--filter-patch]

Example output for bundle outdated --filter-major

Image

Other examples

A useful flag is --strict as it will only list versions that are allowed by your Gemfile requirements (e.g. does not show rails update to 6 if your Gemfile has the line gem 'rails', '~>5.2').

I also experienced that doing upgrades per group (test, development) are easier to do. Thus --groups might also be helpful.

$ bundle...

Some tips for upgrading Bootstrap from 3 to 4

Recently I made an upgrade from Bootstrap 3 to Bootstrap 4 in a bigger project. Here are some tips how to plan and perform such an upgrade. The effort will scale with the size of the project and its structure. If your stylesheets already follow strict rules, it may take less time to adapt them to the new version.

Preparation

There are several gems and libraries that works well with bootstrap or provide at least stylesheets/plugins to easily integrate the bootstrap theme. But very often they only work with specific version or are no long...

How to fix "Command "webpack" not found"

I just ran into this deployment error after switching from the asset pipeline to webpack:

01:05 deploy:assets:precompile
      01 bundle exec rake assets:precompile
      01 Compiling...
      01 Compilation failed:
      01 yarn run v1.22.5
      01 error Command "webpack" not found.
rake stderr: Nothing written

The problem is not related to the "webpack" dependency. You probably just forgot to add a binstub to run "yarn install":

Add these lines to "bin/ya...

Using the Truemail gem to validate e-mail addresses

The Truemail gem (not to be confused with truemail.io) allows validating email addresses, e.g. when users enter them into a sign-up form. It runs inside your application and does not depend on an external SaaS service.

Truemail supports different validation "layers":

  1. Regex validation: if the given address is syntactically valid
  2. DNS validation (called MX validation): if the given domain exists and can receive email
  3. SMTP validation: connects to the host received from DNS and starts a test d...

Capybara: Running tests with headless Chrome

Headless Chrome is a way to run the Chrome browser without a visible window.

Configuring Capybara

Configure the Capybara driver like this:

Capybara.register_driver :selenium do |app|
  options = Selenium::WebDriver::Chrome::Options.new
  options.add_argument('--disable-infobars')
  options.add_emulation(device_metrics: { width: 1280, height: 960, touch: false })
  
  unless ENV.key?('NO_HEADLESS')
    options.add_argument('--headless')
    o...

SSHKit 1.9.0 failure for Capistrano deploy

SSHKit 1.9.0 might fail with the following error, when trying to deploy a Rail application. Upgrading the gem to version 1.21.0 fixed the issue.

Traceback (most recent call last):
	17: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9.0/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
	16: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9.0/lib/sshkit/backends/abstract.rb:29:in `run'
	15: from /home/user/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/sshkit-1.9....

Whenever requires you to set the application attribute in the Capistrano config

Whenever requires you to set the application attribute in your Capistrano configuration. Otherwise your cronjobs are created multiple times.

Example entry in config/deploy.rb:

set :application, 'some-app' # allows "set :whenever_identifier, ->{ "#{fetch(:application)}_#{fetch(:stage)}" }" to work as expected

Good

Then the crontab -l output will look like this:

# Begin Whenever generated tasks for: som...

Missing certificates for rubygems and bundler in Ruby 1.8.7

Using Ruby 1.8.7 you will not be able to use the maximum versions Rubygems 1.8.30 and Bundler 1.17.3 with https://rubygems.org/ anymore. This is a result of a server certificate on December 5th, 2020. The resulting errors will look like following:

  • TypeError: can't modify frozen object
  • Could not verify the SSL certificate for https://rubygems.org/*
  • Bundler::Fetcher::CertificateFailureError: Could not verify the SSL certificate for https://index.rubygems.org/versions.
  • `Error fetching data: hostname was not m...