How to negate scope conditions in Rails

Sometimes you want to find the inverse of an ActiveRecord scope. Depending on what you want to achieve, this is quite easy with Rails 7, and a bit more complicated with Rails 6 and below, or when the inverse scope may contain NULL values. [1]

There are two different ways of "inverting a scope":

As an example, consider the following model.

class User < ApplicationRecord
  scope :admins, -> { where(role: ['admin', 'superuser']) }
  # ...
end

Mathematical NOT

You know this one from basic set theory. It proces the "complementa...

Manage Linux services on the command line (Ubuntu)

Ubuntu 18.04 uses systemd to manage services.

There are basically two commands for listing all services and manipulating the state of a certain service: service and systemctl:

  • service manages System V init scripts
  • systemctl controls the state of the systemd system and service manager. It is backwards compatible to System V and includes the System V services

Therefore I prefer to use systemctl.


See which services are there

>systemctl list-units -a --type=service
  UNIT                                 LOAD     ...

Migrate gem tests from Travis CI to Github Actions with gemika

We currently test most of our gems on Travis CI, but want to migrate those tests to Github Actions. This is a step-by-step guide on how to do this.

Note that this guide requires the gem to use gemika.

  1. Go to a new "ci" branch:
    git checkout -b ci
    
  2. Update gemika to version >= 0.5.0 in all your Gemfiles.
  3. Have gemika generate a Github Actions workflow definition by running
    mkdir -p .github/workflows; bundle exec rake gemika:generate_github_actions_workflow > .github/workf...
    

Debugging SPF records

While debugging a SPF record I found spf-record.de to be very helpful.

  • it lists all IPs that are covered by the SPF record
  • shows syntax errors
  • helps you debugging errors like DNS lookup limit reached
  • it also lets you test a new SPF strings before applying it. This can save you time as you don't have to loop with operations

Also the advanced check at vamsoft.com has a very good interface to test new SPF policies.

PostgreSQL: Importing dumps created with newer versions

When loading a database dump created with pg_dump into your database, you might run into an error like

pg_restore: error: unsupported version (1.15) in file header

This is because your local pg_restore version is too old to match the format created by pg_dump. The version of the PostgreSQL server doesn't matter here.

For example, the official Ubuntu 20.04 sources include only PostgreSQL 12, so your pg_restore version will also be v12. Ubuntu 22.04 includes version 14 in its sources.
Both seem to be incompatible with dumps ...

Rails: How to restore a postgres dump from the past

It sometimes happen that a database dump, that would want to insert into your development database, does not match the current schema of the database. This often happens when you have an old dump, but your current setup is up to date with the the master.

Hint: In most cases it is sufficient to delete and recreate the local database in order to import the dump. If any problems occur, proceed as follows:

1. Figure out the original migration status of the dumpfile

  • Convert your dump to plaintext: `pg_restore -f some.dump > some.dump....

How to implement simple queue limiting/throttling for Sidekiq

The sidekiq-rate-limiter gem allows rate-limiting Sidekiq jobs and works like a charm. However, it needs to be integrated on a per-worker basis.

If you want to limit a whole queue instead, and if your requirements are simple enough, you can do it via a Sidekiq middleware yourself.

Here is an example that limits concurrency of the "mailers" queue to 1. It uses a database mutex via the [with_advisory_lock](https://github.com/ClosureTree/wit...

VCR and the webdrivers gem

If you're using the webdrivers gem and VCR together, depending on your configuration, VCR will yell at you regulary.
The webdrivers gem tries to update your webdrivers on your local machine. To do so, it checks the internet for newer versions, firing an HTTP-request to e.g. https://chromedriver.storage.googleapis.com

You can "fix" this in multiple ways:

  1. Update your drivers on your machine with
    RAILS_ENV=test rake webdrivers:chromedriver:update

  2. Ignore the driver update-URL in your ...

Simple form examples with bootstrap

Good reference how to build bootstrap forms with simple_form.

How to include Sidekiq job IDs in Rails logs

When logging in Rails, you can use the log_tags configuration option to add extra information to each line, like :request_id or :subdomain. However, those are only valid inside a request context and have no effect when your application is logging from inside a Sidekiq process.
This includes custom as well as any framework logs, like query logging from ActiveRecord.

Since Sidekiq Workers run inside threads of a single process, running multiple jobs in...

How to generate GIDs from an ActiveRecord scope

ActiveRecord provides the ids method to pluck ids from a scope, but what if you need to pluck Global IDs?

While you could just call map(&:to_global_id) on your scope, this approach would instantiate each record just to do that. When you have many records, this will at the very least be slow.

Here is a method that does it for you efficiently. It respects Single Table Inheritance (STI).
Put it in your project's ApplicationRecord to make it available on all models.

class ApplicationRecord
  ...

Ruby: How to determine the absolute path relative to a file

If you want to get the path of a file relative to another, you can use the expand_path method with either the constant __FILE__ or the method __dir__. Read this card for more information about __FILE__ and __dir__.

Example

Structure:

.
├── bin
│   ├── format_changelog
├── CHANGELOG.md

bin/format_changelog:

#!/usr/bin/env ruby

changelog_path = ? # How to get the path to ../CHANGELOG.md independent of the working dir of the caller
changelog = File.read(changelog_path)

# ... further actions...

Chrome: Using browser notifications

Development

Google Chrome disables Notifications for insecure origins (i.e. those using HTTP). Only http://localhost is considered secure.

If you need to use browser notifications on other origins, you can set a flag: chrome://flags/#unsafely-treat-insecure-origin-as-secure. Enable the flag and add your origins. Remember that "origin" refers to the combination of protocol+hostname+port, e.g. "http://example.com:8088".

CSS variables aka CSS Custom Properties

CSS variables are very different from preprocessor variables. While preprocessors use variables to compile a static piece of CSS, CSS custom properties are a reactive (i.e. live) part of the styles. Think of them like usual CSS properties that cascade, but have:

  • A special syntax: CSS variables always start with a double-dash (--color)
  • No inherent meaning: Defining a CSS variable will not change any styles in itself
  • A special functionality: CSS variables can be used within the values of other properties, including CSS variables...

Controlling issue grouping in Sentry

When you use Sentry to monitor exceptions, an important feature is Sentry's error grouping mechanism. It will aggregate similar error "events" into one issue, so you can track and monitor it more easily. Grouping is especially important when you try to silence certain errors.

It is worth understanding how Sentry's grouping mechanism works.

The default grouping mechanism

The exact algorithm has changed over time, and Sentry will keep using the algorithm t...

Clean your Rails routes: grouping

In Ruby on Rails, all the routes of a given application can be found within the config/routes.rb file.
You add more and more routes in this file as your project grows.

The problem here is that this file potentially becomes very complicated to manage over the time.
That’s why it’s important to find a way to order and maintain your routes.

See: Clean your Rails routes: grouping

Sometimes the routes.rb grows very fast and each line adds mo...

Ruby: Appending lines to a file in sync

When writing some logs to a file, that don't use Ruby's logger utility, it is often useful to sync them. So other process can read the output just in time.

Example with enabled sync

log_path = '/tmp/some_log.log'

log_file = File.open(log_path, 'a+')
log_file.sync = true

log_file.puts('Some log message')
File.read(log_path) #=> "Some log message\n"

log_file.puts('Some other message')
File.read(log_path) #=> "Some log message\nSome other message\n"

Example ...

Parallel cucumber: How to pass in cucumber arguments

Here is an example with the --tags option. You need to wrap them inside --cucumber-options option of parallel_cucumber.

DISPLAY=:17 bundle exec parallel_cucumber --cucumber-options '--tags @solo' features

See more details in the docs.

Rails: How to get the ordered list of used middlewares

Rails middlewares are small code pieces that wrap requests to the application. The first middleware gets passed the request, invokes the next, and so on. Finally, the application is invoked, builds a response and passes it back to the last middleware. Each middleware now returns the response until the request is answered. Think of it like Russian Dolls, where each middleware is a doll and the application is the innermost item.

You can run rake middleware to get the ordered list of used middlewares in a Rails application:

$> rake midd...

How to check if a file is a human readable text file

Ruby's File class has a handy method binary? which checks whether a file is a binary file. This method might be telling the truth most of the time. But sometimes it doesn't, and that's what causes pain. The method is defined as follows:

# Returns whether or not +file+ is a binary file.  Note that this is
# not guaranteed to be 100% accurate.  It performs a "best guess" based
# on a simple test of the first +File.blksize+ characters.
#
# Example:
#
#   File.binary?('somefile.exe') # => true
#   File.binary?('somefile.txt') # => fal...

PostgreSQL: How to use with_advisory_lock to prevent race conditions

If you want to prevent that two processes run some code at the same time you can use the gem with_advisory_lock.

What happens

  1. The thread will wait indefinitely until the lock is acquired.
  2. While inside the block, you will exclusively own the advisory lock.
  3. The lock will be released after your block ends, even if an exception is raised in the block.

This is usually required if there is no suitable database row to lock on.

Example

You want to generate a...

How to cycle through grep results with vim

grep is the go-to CLI tool to accomplish tasks like filtering large files for arbitrary keywords. When additional context is needed for search results, you might find yourself adding flags like -B5 -A10 to your query. Now, every search result covers 16 lines of your bash.

There is another way: You can easily pipe your search results to the VIM editor and cycle through them.

Example: Searching for local occurrences of "User"

vim -q <(grep -Hn -r "User" .)

# vim -q starts vim in the "quickfix" mode. See ":help quickfix"
# grep...

How to migrate CoffeeScript files from Sprockets to Webpack(er)

If you migrate a Rails application from Sprockets to Webpack(er), you can either transpile your CoffeeScript files to JavaScript or integrate a CoffeeScript compiler to your new process. This checklist can be used to achieve the latter.

  1. If you need to continue exposing your CoffeeScript classes to the global namespace, define them on window directly:
-class @User
+class window.User
  1. Replace Sprocket's require statement with Webpacker's...

Vortrag: Content Security Policy: Eine Einführung

Grundidee

CSP hat zum Ziel einen Browser-seitigen Mechanismus zu schaffen um einige Angriffe auf Webseiten zu verhindern, hauptsächlich XSS-Angriffe.

Einschub: Was ist XSS?

XSS = Cross Site Scripting. Passiert wenn ein User ungefiltertes HTML in die Webseite einfügen kann.

<div class="comment">
  Danke für den interessanten Beitrag! <script>alert('you have been hacked')</script>
</div>

Rails löst das Problem weitgehend, aber

  • Programmierfehler weiter möglich
  • manchmal Sicherheitslücken in Gems oder Rails

Lösungsid...