How to include Sidekiq job IDs in Rails logs

When logging in Rails, you can use the log_tags configuration option to add extra information to each line, like :request_id or :subdomain. However, those are only valid inside a request context and have no effect when your application is logging from inside a Sidekiq process.
This includes custom as well as any framework logs, like query logging from ActiveRecord.

Since Sidekiq Workers run inside threads of a single process, running multiple jobs in...

How to generate GIDs from an ActiveRecord scope

ActiveRecord provides the ids method to pluck ids from a scope, but what if you need to pluck Global IDs?

While you could just call map(&:to_global_id) on your scope, this approach would instantiate each record just to do that. When you have many records, this will at the very least be slow.

Here is a method that does it for you efficiently. It respects Single Table Inheritance (STI).
Put it in your project's ApplicationRecord to make it available on all models.

class ApplicationRecord
  ...

Ruby: How to determine the absolute path relative to a file

If you want to get the path of a file relative to another, you can use the expand_path method with either the constant __FILE__ or the method __dir__. Read this card for more information about __FILE__ and __dir__.

Example

Structure:

.
├── bin
│   ├── format_changelog
├── CHANGELOG.md

bin/format_changelog:

#!/usr/bin/env ruby

changelog_path = ? # How to get the path to ../CHANGELOG.md independent of the working dir of the caller
changelog = File.read(changelog_path)

# ... further actions...

Chrome: Using browser notifications

Development

Google Chrome disables Notifications for insecure origins (i.e. those using HTTP). Only http://localhost is considered secure.

If you need to use browser notifications on other origins, you can set a flag: chrome://flags/#unsafely-treat-insecure-origin-as-secure. Enable the flag and add your origins. Remember that "origin" refers to the combination of protocol+hostname+port, e.g. "http://example.com:8088".

CSS variables aka CSS Custom Properties

CSS variables are very different from preprocessor variables. While preprocessors use variables to compile a static piece of CSS, CSS custom properties are a reactive (i.e. live) part of the styles. Think of them like usual CSS properties that cascade, but have:

  • A special syntax: CSS variables always start with a double-dash (--color)
  • No inherent meaning: Defining a CSS variable will not change any styles in itself
  • A special functionality: CSS variables can be used within the values of other properties, including CSS variables...

Controlling issue grouping in Sentry

When you use Sentry to monitor exceptions, an important feature is Sentry's error grouping mechanism. It will aggregate similar error "events" into one issue, so you can track and monitor it more easily. Grouping is especially important when you try to silence certain errors.

It is worth understanding how Sentry's grouping mechanism works.

The default grouping mechanism

The exact algorithm has changed over time, and Sentry will keep using the algorithm t...

Clean your Rails routes: grouping

In Ruby on Rails, all the routes of a given application can be found within the config/routes.rb file.
You add more and more routes in this file as your project grows.

The problem here is that this file potentially becomes very complicated to manage over the time.
That’s why it’s important to find a way to order and maintain your routes.

See: Clean your Rails routes: grouping

Sometimes the routes.rb grows very fast and each line adds mo...

Ruby: Appending lines to a file in sync

When writing some logs to a file, that don't use Ruby's logger utility, it is often useful to sync them. So other process can read the output just in time.

Example with enabled sync

log_path = '/tmp/some_log.log'

log_file = File.open(log_path, 'a+')
log_file.sync = true

log_file.puts('Some log message')
File.read(log_path) #=> "Some log message\n"

log_file.puts('Some other message')
File.read(log_path) #=> "Some log message\nSome other message\n"

Example ...

Parallel cucumber: How to pass in cucumber arguments

Here is an example with the --tags option. You need to wrap them inside --cucumber-options option of parallel_cucumber.

DISPLAY=:17 bundle exec parallel_cucumber --cucumber-options '--tags @solo' features

See more details in the docs.

Rails: How to get the ordered list of used middlewares

Rails middlewares are small code pieces that wrap requests to the application. The first middleware gets passed the request, invokes the next, and so on. Finally, the application is invoked, builds a response and passes it back to the last middleware. Each middleware now returns the response until the request is answered. Think of it like Russian Dolls, where each middleware is a doll and the application is the innermost item.

You can run rake middleware to get the ordered list of used middlewares in a Rails application:

$> rake midd...

How to check if a file is a human readable text file

Ruby's File class has a handy method binary? which checks whether a file is a binary file. This method might be telling the truth most of the time. But sometimes it doesn't, and that's what causes pain. The method is defined as follows:

# Returns whether or not +file+ is a binary file.  Note that this is
# not guaranteed to be 100% accurate.  It performs a "best guess" based
# on a simple test of the first +File.blksize+ characters.
#
# Example:
#
#   File.binary?('somefile.exe') # => true
#   File.binary?('somefile.txt') # => fal...

PostgreSQL: How to use with_advisory_lock to prevent race conditions

If you want to prevent that two processes run some code at the same time you can use the gem with_advisory_lock.

What happens

  1. The thread will wait indefinitely until the lock is acquired.
  2. While inside the block, you will exclusively own the advisory lock.
  3. The lock will be released after your block ends, even if an exception is raised in the block.

This is usually required if there is no suitable database row to lock on.

Example

You want to generate a...

How to cycle through grep results with vim

grep is the go-to CLI tool to accomplish tasks like filtering large files for arbitrary keywords. When additional context is needed for search results, you might find yourself adding flags like -B5 -A10 to your query. Now, every search result covers 16 lines of your bash.

There is another way: You can easily pipe your search results to the VIM editor and cycle through them.

Example: Searching for local occurrences of "User"

vim -q <(grep -Hn -r "User" .)

# vim -q starts vim in the "quickfix" mode. See ":help quickfix"
# grep...

How to migrate CoffeeScript files from Sprockets to Webpack(er)

If you migrate a Rails application from Sprockets to Webpack(er), you can either transpile your CoffeeScript files to JavaScript or integrate a CoffeeScript compiler to your new process. This checklist can be used to achieve the latter.

  1. If you need to continue exposing your CoffeeScript classes to the global namespace, define them on window directly:
-class @User
+class window.User
  1. Replace Sprocket's require statement with Webpacker's...

Vortrag: Content Security Policy: Eine Einführung

Grundidee

CSP hat zum Ziel einen Browser-seitigen Mechanismus zu schaffen um einige Angriffe auf Webseiten zu verhindern, hauptsächlich XSS-Angriffe.

Einschub: Was ist XSS?

XSS = Cross Site Scripting. Passiert wenn ein User ungefiltertes HTML in die Webseite einfügen kann.

<div class="comment">
  Danke für den interessanten Beitrag! <script>alert('you have been hacked')</script>
</div>

Rails löst das Problem weitgehend, aber

  • Programmierfehler weiter möglich
  • manchmal Sicherheitslücken in Gems oder Rails

Lösungsid...

Sentry: Different ways of deferring notifications for an issue

We use Sentry to be informed about different kinds of issues. One of the key features is that you are not getting spammed if many errors of the same kind occur in a small timespan.
If an issue pops up the usual workflow is to fix the code and mark the issue as "resolved" in Sentry. Only new or resolved issues trigger another email notification when they are proxied through Sentry.

This workflow does not fit well for issues we cannot fix, e.g. when consuming an external API that is sometimes down. In such cases you ...

Automatically validating dependency licenses with License Finder

"Open-source software (OSS) is great. Anyone can use virtually any open-source code in their projects."

Well, it depends. Licenses can make things difficult, especially when you are developing closed-source software. Since some OSS licenses even require the employing application to be open-sourced as well (looking at you, GPL), you cannot use such software in a closed-source project.

To be sure on this, we have developed a project-level integration of Pivotal's excellent [license_finder](https:/...

ActiveRecord: String and text fields should always validate their length

If you have a :string or :text field, you should pair it with a model validation that restricts its length.

There are two motivations for this:

  • In modern Rails, database types :string and :text no longer have a relevant size limit. Without a validation a malicious user can quickly exhaust the hard drive of your database server.
  • In legacy Rails (or database schemas migrated from legacy Rails), database types :string and :text had a database-side length constraint. When the user enters a longer string, the ActiveRecord valida...

How to prevent Nokogiri from fixing invalid HTML

Nokogiri is great. It will even fix invalid HTML for you, like a browser would (e.g. move block elements out of parents which are specified to not allow them).

>> Nokogiri::HTML.fragment("<h1><p>foo</p><span>bar</span></h1>").to_s
=> "<h1></h1><p>foo</p><span>bar</span>"

While this is mostly useful, browsers are actually fine with a bit of badly formatted HTML. And you don't want to be the one to blame when the SEO folks complain about an empty <h1>.

To avoid said behavior, use Nokogiri::XML instead of Nokogiri::HTML whe...

FactoryBot: Traits for enums

FactoryBot allows to create traits from Enums since version 6.0.0

The automatic definition of traits for Active Record enum attributes is enabled by default, for non-Active Record enums you can use the traits_for_enum method.

Example

factory :user do
  traits_for_enum :role, %w[admin contact] # you can use User::ROLES here, of course
end

is equivalent to

factory :user do
  trait :admin do
    role { 'admin' }
  end

  trait :contact do
    role { 'c...

The HTML5 video element

# Basic HTML example
<video poster="preview_image.png" controls>
  <source src="or_here.webm" type="video/webm" />
  <source src="alternative_if_browser_cant_pay_first_source.mp4" type="video/mp4" />
  <track src="optional_subtitles.vtt" kind="subtitles" srclang="de" label="Deutsch" default>
</video>

# Javascript API (notable methods and properties)
video = document.querySelector('video')
video.play()
video.pause()
video.load() // Reset to the beginning and select the best available source
video.currentSrc // The selected source
video.c...

HTTP Client in RubyMine

RubyMine has a HTTP Client that can be useful to test web APIs.
Just create a .http scratch file an write your request in it.
The request can then be executed with the "Run all requests in File" button above the file.

Some alternatives:

The format for request is like this:

Method Request-URI HTTP-Version
Header-field: Heade...

How to use Active Job to decouple your background processing from a gem

In a web application you sometimes have tasks that can not be processed during a request but need to go to the background.
There are several gems that help to you do that, like Sidekiq or Resque.

With newer Rails you can also use ActiveJob as interface for a background processing library. See here for a list of supported queueing adapters.
For ...

Testing for XSS in Markdown Fields

If you render markdown from user input, an attacker might be able to use this to inject javascript code into the source code of your page.
The linked github page is a collection of common markdown XSS payloads which is handy for writing tests.

Producing arbitrary links:

[Basic](javascript:alert('Basic'))
[Local Storage](javascript:alert(JSON.stringify(localStorage)))
[CaseInsensitive](JaVaScRiPt:alert('CaseInsensitive'))
[URL](javascript://www.google.com%0Aalert('URL'))
[In Quotes]('javascript:alert("InQuotes")')

Using onload...