Shell script to magically configure display setup
Here is a bash script that I use to auto-configure displays on Ubuntu 24.04 with Xorg.
Background
- Ubuntu always sets the primary display to the 1st (i.e. internal) display whenever I connect to a new Dock/Hub.
- I want my primary display to be the large display.
- My notebook is always placed left of external displays, so the 2nd display will be the center (or only) external display and should be primary.
- I also want all my displays to be placed horizontally, but bottom-aligned (the default would be aligned at their top edges)....
Debugging Capistrano
Capistrano 3 has a doctor
task that will print information about
- Environment: Ruby, Rubygems and Bundler versions
- List of Capistrano gems and whether an update is available
- All config variables and their values
- Capistrano server config
$ bundle exec cap staging doctor
Ruby: Retrieving and processing files via Selenium and JavaScript
This card shows an uncommon way to retrieve a file using selenium where JavaScript is used to return a binary data array to Ruby code.
The following code example retrieves a PDF but the approach also works for other file types.
require "selenium-webdriver"
selenium_driver = Selenium::WebDriver.for :chrome
selenium_driver.navigate.to('https://example.com')
link_to_pdf = 'https://blobs.example.com/random-pdf'
binary_data_array = selenium_driver.execute_script(<<-JS, link_to_pdf)
const response = await fetch(arguments[0])
if (!r...
RSpec: Applying stubs only within a block
When you mocked method calls in RSpec, they are mocked until the end of a spec, or until you explicitly release them.
You can use RSpec::Mocks.with_temporary_scope
to have all mocks applied inside a block to be released when the block ends.
Example:
RSpec::Mocks.with_temporary_scope do
allow(Rails).to receive(:env).and_return('production'.inquiry)
puts Rails.env # prints "production"
end
puts Rails.env # prints "test"
Note that, when overriding pre-existing mocks inside the block, they are not reverted to the previously ...
Best practices: Writing a Rails script (and how to test it)
A Rails script lives in lib/scripts and is run with bin/rails runner lib/scripts/...
. They are a simple tool to perform some one-time actions on your Rails application. A Rails script has a few advantages over pasting some prepared code into a Rails console:
- Version control
- Part of the repository, so you can build on previous scripts for a similar task
- You can have tests (see below)
Although not part of the application, your script is code and should adhere to the common quality standards (e.g. no spaghetti code). However, a script...
Bash: How to count and sort requests by IP from the access logs
Example
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.41 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
Goal
Count and sort the number of requests for a single IP address.
Bash Command
awk '{ print $1}' test.log | sort...
Bash: How to grep logs for a pattern and expand it to the full request
Example
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [53a240c1-489e-4936-bbeb-d6f77284cf38] nope
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- ...
Opening a zipped coverage report with one click
Tested on Ubunut 22.04
1. Opener script
- Create a file
~/.local/bin/coverage_zip_opener
with:
#!/bin/bash
tmp_folder="/tmp/coverage-report-opener"
if [ -z "$1" ]
then
echo "Usage: coverage_zip_opener [filename]"
exit -1
fi
if ! [[ "$1" =~ ^.*Pipeline.*Coverage.*\.zip$ || "$1" =~ ^.*merged_coverage_report.*\.zip$ ]]; then
file-roller "$1"
exit 0
fi
rm -Rf $tmp_folder
unzip -qq "$1" -d $tmp_folder
index_filename=$(find /tmp/coverage-report-opener -name "index.html" | ...
open-next-failure: An alias to speed up test debugging
Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.
The following bash aliases helped me speed up that process:
alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"
There is a lot going on above but the gist...
Capistrano 3: Running a command on all servers
This Capistrano task runs a command on all servers.
bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
Code
# lib/capistrano/tasks/app.rake
namespace :app do
# Use e.g. to grep logs on all servers:
# b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
#
# * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
# * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
# * To be able to process ...
Debug MiniMagick calls in your Rails app
Most of our applications use CarrierWave for file uploads. CarrierWave has an integrated processing mechanism for different file versions with support for ImageMagick through CarrierWave::MiniMagick
(which requires the mini_magick
gem). In case your processing runs into an error, CarrierWave will just swallow it and rethrow an error with a very generic message like Processing failed. Maybe it is not an image?
which does not help you finding out what the actual problem is. CarrierWave probably does this for security purposes, but does n...
How to make sure that manual deploy tasks (scheduled in Pivotal Tracker) are executed on deploy (with Capistrano)
We regularly have tasks that need to be performed around a deploy. Be it to notify operations about changed application behavior, be it to run a little oneline script after the deploy. Most database-related stuff can be handled by migrations, but every once in a while, we have tasks that are much easier to be performed manually.
Writing deploy tasks
Here is how we manage the deploy tasks themselves:
- Deploy tasks are written inside the Pivotal Tracker story description, clearly marked (e.g. with a headline "Deploy task")
- We disting...
CarrierWave: Processing images with libvips
When you write your next CarrierWave uploader, consider processing your images with libvips instead of ImageMagick.
Reasons for libvips
There are several upsides to using libvips over ImageMagick:
- libvips is considerably faster and uses less memory.
- ImageMagick has a large attack surface that has repeatedly caused security incidents in the past (compare [ImageMagick CVEs](https://www....
How to pretty print all values in a Redis database
With this Ruby script you can print all values in a Redis database to your console (derived from this bash script).
Note: Do not run this command in production. This is for debugging purposes only.
def pretty_print_redis(redis)
redis.keys.each_with_object({}) do |key, hash|
type = redis.type(key)
hash[key] = case type
when 'string'
redis.get(key)
when 'hash'
redis.hgetall(key)
when 'list'
redis.lrange(key, 0, -1)
when 'set'
redis.smembers(...
Sidekiq 7: Rate limiting with capsules
Sidekiq 7 adds a new feature called capsules.
Use cases:
- a
chrome
queue limited to1
for e.g. PDF processing to not overload the application server - an
api
queue, that limits a queue to2
to protect the API server from too many requests in parallel
Example:
Sidekiq.configure_server do |config|
# Edits the default capsule
config.queues = %w[critical default low]
config.concurrency = 5
# Define a new capsule which ...
Issue Checklist Template
This is a checklist I use to work on issues. For this purpose I extracted several cards related to the makandra process and ported them into a check list and refined that over time a little bit.
This task list is divided by the Gate keeping process in the following steps:
1. Starting a new feature
2. Working on the issue
3. Finishing a feature
4. After Review
Here are some ti...
Git: Restore
tl;dr
git checkout
is the swiss army of git commands. If you prefer a semantically more meaningful command for restoring tasks, usegit restore
instead.With this command you can ...
- ... do unstaging -
git restore --staged
- ... discard staged changes -
git restore --staged --worktree
- ... discard unstaged changes -
git restore
- ... restore deleted files -
git restore
- ... restore historic versions -
git restore --source
- ... recreate merge conflicts -
git restore --merge
- ... specifiy...
How to fix "Exit with code 1 due to network error: ProtocolUnknownError" with wkhtmltopdf
New versions of wkhtmltopdf dissallow file://
URLs by default. You can allow them by passing --enable-local-file-access
.
If you are using PDFKit, set the option
PDFKit.configure do |config|
config.default_options = {
enable_local_file_access: true,
}
end
This will be necessary in many setups to allow wkhtmltopdf to fetch assets (such as stylesheets) from the filesystem.
Note on security
Allowing this poses some risk when you render user input, since it might be feasible to include data from the local filesyste...
Converting SVG to other vector formats without Inkscape
If you need to convert an SVG source to PS or EPS, the most common suggestion on the interwebs is to use Inkscape from the commandline.
Inkscape is a fairly resource-heavy tool with lots of dependencies. A great alternative for converting is CairoSVG.
CairoSVG is available on most Linux distros through their package management systems, e.g. apt install cairosvg
on Ubuntu.
It has few dependencies (most importantly Python 3 and some related packages, but really not much)...
How to debug issues with zeitwerk and Rails
In case you have trouble with the zeitwerk autoloader, you can check out the documentation Autoloading and Reloading Constants and Classic to Zeitwerk HOWTO for some debugging hints.
For myself it was useful to print the registered constants and the file references during the boot. Therefore you need to add Rails.autoloaders.log!
at the end of your config/application.rb
file. You could also run `bin...
How to get information about a gem (via CLI or at runtime from Ruby)
When you need information about a gem (like version(s) or install path(s)), you can use the gem
binary from the command line, or the Gem
API inside a ruby process at runtime.
gem
binary (in a terminal)
You can get some information about a gem by running gem info <gem name>
in your terminal.
Example:
$ gem info irb
*** LOCAL GEMS ***
irb (1.4.1, 1.3.5)
Author: Keiju ISHITSUKA
Homepage: https://github.com/ruby/irb
Licenses: Ruby, BSD-2-Clause
Installed at (1.4.1): /home/arne/.rbenv/versions/3.0.3/lib/ruby/g...
Generating an Entity Relationship Diagram for your Rails application
This card explains how to generate an entity relationship diagram for your Rails application.
We also show how to limit your ERD to a subset of models, e.g. models inside a namespace.
Generating a full ERD
Option A: RubyMine
- Right-click anywhere in your project tree
- In the context menu, find the "Diagrams" menu item at/near the bottom
- Inside, choose "Show diagram" → "Rails Model Dependency Diagram"
- A new tab will open with the diagram inside. You can modify it there, and export it as an image.
Option B: Use rails-e...
Using feature flags to stabilize flaky E2E tests
A flaky test is a test that is often green, but sometimes red. It may only fail on some PCs, or only when the entire test suite is run.
There are many causes for flaky tests. This card focuses on a specific class of feature with heavy side effects, mostly on on the UI. Features like the following can amplify your flakiness issues by unexpectedly changing elements, causing excessive requests or other timing issues:
- Lazy loading images
- Autocomplete in search f...
Git shortcut to rebase onto another branch
Inspired by recent "git shortcut" cards I figured it would be nice to have one of these for rebasing a few commits onto another branch. The usual notation is prone to of-by-one errors as you have to either specify the commit before the ones you want to move or count the number of commits.
You may add this rebase-onto
function to your ~/.bashrc
:
function rebase-onto {
commit=$(git log --oneline | fzf --prompt 'Select the first commit y...