Best practices: Writing a Rails script (and how to test it)
A Rails script lives in lib/scripts and is run with bin/rails runner lib/scripts/...
. They are a simple tool to perform some one-time actions on your Rails application. A Rails script has a few advantages over pasting some prepared code into a Rails console:
- Version control
- Part of the repository, so you can build on previous scripts for a similar task
- You can have tests (see below)
Although not part of the application, your script is code and should adhere to the common quality standards (e.g. no spaghetti code). However, a script...
Bash: How to count and sort requests by IP from the access logs
Example
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.41 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
Goal
Count and sort the number of requests for a single IP address.
Bash Command
awk '{ print $1}' test.log | sort...
Bash: How to grep logs for a pattern and expand it to the full request
Example
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [53a240c1-489e-4936-bbeb-d6f77284cf38] nope
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- ...
open-next-failure: An alias to speed up test debugging
Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.
The following bash aliases helped me speed up that process:
alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"
There is a lot going on above but the gist...
Capistrano 3: Running a command on all servers
This Capistrano task runs a command on all servers.
bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
Code
# lib/capistrano/tasks/app.rake
namespace :app do
# Use e.g. to grep logs on all servers:
# b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
#
# * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
# * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
# * To be able to process ...
Debug MiniMagick calls in your Rails app
Most of our applications use CarrierWave for file uploads. CarrierWave has an integrated processing mechanism for different file versions with support for ImageMagick through CarrierWave::MiniMagick
(which requires the mini_magick
gem). In case your processing runs into an error, CarrierWave will just swallow it and rethrow an error with a very generic message like Processing failed. Maybe it is not an image?
which does not help you finding out what the actual problem is. CarrierWave probably does this for security purposes, but does n...
How to make sure that manual deploy tasks (scheduled in Pivotal Tracker) are executed on deploy (with Capistrano)
We regularly have tasks that need to be performed around a deploy. Be it to notify operations about changed application behavior, be it to run a little oneline script after the deploy. Most database-related stuff can be handled by migrations, but every once in a while, we have tasks that are much easier to be performed manually.
Writing deploy tasks
Here is how we manage the deploy tasks themselves:
- Deploy tasks are written inside the Pivotal Tracker story description, clearly marked (e.g. with a headline "Deploy task")
- We disting...
CarrierWave: Processing images with libvips
When you write your next CarrierWave uploader, consider processing your images with libvips instead of ImageMagick.
Reasons for libvips
There are several upsides to using libvips over ImageMagick:
- libvips is considerably faster and uses less memory.
- ImageMagick has a large attack surface that has repeatedly caused security incidents in the past (compare [ImageMagick CVEs](https://www....
How to pretty print all values in a Redis database
With this Ruby script you can print all values in a Redis database to your console (derived from this bash script).
Note: Do not run this command in production. This is for debugging purposes only.
def pretty_print_redis(redis)
redis.keys.each_with_object({}) do |key, hash|
type = redis.type(key)
hash[key] = case type
when 'string'
redis.get(key)
when 'hash'
redis.hgetall(key)
when 'list'
redis.lrange(key, 0, -1)
when 'set'
redis.smembers(...
Sidekiq 7: Rate limiting with capsules
Sidekiq 7 adds a new feature called capsules.
Use cases:
- a
chrome
queue limited to1
for e.g. PDF processing to not overload the application server - an
api
queue, that limits a queue to2
to protect the API server from too many requests in parallel
Example:
Sidekiq.configure_server do |config|
# Edits the default capsule
config.queues = %w[critical default low]
config.concurrency = 5
# Define a new capsule which ...
Sass: get rid of deprecation warnings in dependencies
TLDR: sass >= 1.35.0
has the option quietDeps
to silence deprecation warnings from dependencies.
quietDeps: If true, the compiler must not print deprecation warnings
for stylesheets that are transitively loaded through an import path or importer.
You might have seen deprecation warnings like this during assets compilation:
DEPRECATION WARNING: Using / for division is deprecated and will be removed in Dart Sass 2.0.0.
Recommendation: math.div($grid-gutter-width, 2)
More info and automated migrator: https://sass-la...
Git: Restore
tl;dr
git checkout
is the swiss army of git commands. If you prefer a semantically more meaningful command for restoring tasks, usegit restore
instead.With this command you can ...
- ... do unstaging -
git restore --staged
- ... discard staged changes -
git restore --staged --worktree
- ... discard unstaged changes -
git restore
- ... restore deleted files -
git restore
- ... restore historic versions -
git restore --source
- ... recreate merge conflicts -
git restore --merge
- ... specifiy...
How to fix "Exit with code 1 due to network error: ProtocolUnknownError" with wkhtmltopdf
New versions of wkhtmltopdf dissallow file://
URLs by default. You can allow them by passing --enable-local-file-access
.
If you are using PDFKit, set the option
PDFKit.configure do |config|
config.default_options = {
enable_local_file_access: true,
}
end
This will be necessary in many setups to allow wkhtmltopdf to fetch assets (such as stylesheets) from the filesystem.
Note on security
Allowing this poses some risk when you render user input, since it might be feasible to include data from the local filesyste...
Converting SVG to other vector formats without Inkscape
If you need to convert an SVG source to PS or EPS, the most common suggestion on the interwebs is to use Inkscape from the commandline.
Inkscape is a fairly resource-heavy tool with lots of dependencies. A great alternative for converting is CairoSVG.
CairoSVG is available on most Linux distros through their package management systems, e.g. apt install cairosvg
on Ubuntu.
It has few dependencies (most importantly Python 3 and some related packages, but really not much)...
How to debug issues with zeitwerk and Rails
In case you have trouble with the zeitwerk autoloader, you can check out the documentation Autoloading and Reloading Constants and Classic to Zeitwerk HOWTO for some debugging hints.
For myself it was useful to print the registered constants and the file references during the boot. Therefore you need to add Rails.autoloaders.log!
at the end of your config/application.rb
file. You could also run `bin...
How to get information about a gem (via CLI or at runtime from Ruby)
When you need information about a gem (like version(s) or install path(s)), you can use the gem
binary from the command line, or the Gem
API inside a ruby process at runtime.
gem
binary (in a terminal)
You can get some information about a gem by running gem info <gem name>
in your terminal.
Example:
$ gem info irb
*** LOCAL GEMS ***
irb (1.4.1, 1.3.5)
Author: Keiju ISHITSUKA
Homepage: https://github.com/ruby/irb
Licenses: Ruby, BSD-2-Clause
Installed at (1.4.1): /home/arne/.rbenv/versions/3.0.3/lib/ruby/g...
Generating an Entity Relationship Diagram for your Rails application
This card explains how to generate an entity relationship diagram for your Rails application.
We also show how to limit your ERD to a subset of models, e.g. models inside a namespace.
Generating a full ERD
Option A: RubyMine
- Right-click anywhere in your project tree
- In the context menu, find the "Diagrams" menu item at/near the bottom
- Inside, choose "Show diagram" → "Rails Model Dependency Diagram"
- A new tab will open with the diagram inside. You can modify it there, and export it as an image.
Option B: Use rails-e...
Using feature flags to stabilize flaky E2E tests
A flaky test is a test that is often green, but sometimes red. It may only fail on some PCs, or only when the entire test suite is run.
There are many causes for flaky tests. This card focuses on a specific class of feature with heavy side effects, mostly on on the UI. Features like the following can amplify your flakiness issues by unexpectedly changing elements, causing excessive requests or other timing issues:
- Lazy loading images
- Autocomplete in search f...
Git shortcut to rebase onto another branch
Inspired by recent "git shortcut" cards I figured it would be nice to have one of these for rebasing a few commits onto another branch. The usual notation is prone to of-by-one errors as you have to either specify the commit before the ones you want to move or count the number of commits.
You may add this rebase-onto
function to your ~/.bashrc
:
function rebase-onto {
commit=$(git log --oneline | fzf --prompt 'Select the first commit y...
Git shortcut to fixup a recent commit
git --fixup
is very handy to amend a change to a previous commit. You can then autosquash your commits with git rebase -i --autosquash
and git will do the magic for you and bring them in the right order. However, as git --fixup
wants a ref to another commit, it is quite annoying to use since you always have to look up the sha of the commit you want to amend first.
Inspired by the [shortcut to checkout recent branches with fzf](https://makandracards.com/makandra/505126-g...
NVM: How to automatically switch version when changing directories
The Node Version Manager allows installing multiple NodeJS versions and switching between them.
By default, it does not automatically switch versions when entering a directory that holds a .nvmrc
file.
The project's readme document offers a bash function which calls nvm use
after each cd
. In fact, it replaces cd
in your bash.
I did not want to do that, but instead use the $PROMPT_COMMAND
feature. So here is my take on it.
Note that it is much shorter, it probably does a f...
How to generate and test a htpasswd password hash
Generate a password
htpasswd -Bn firstname.lastname
This will ask you for a password and use bcrypt (-B
, more secure) and print the output to stdout (-n
).
Check if password matches the hash
You'll first have to write the password hash to a file:
echo firstname.lastname:$2y$05$4JXxd2GM/J2...9c3KJmFS > htpass_test
Check, if it is correct:
htpasswd -v htpass_test firstname.lastname
You probably should not use the -b
switch to read the password from the command line as the password will then be visible...
Delivering Carrierwave attachments to authorized users only
Preparation
To attach files to your records, you will need a new database column representing the filename of the file. To do this, add a new migration (rails g migration <name>
) with the following content:
class AddAttachmentToNotes < ActiveRecord::Migration[6.0]
def change
add_column :notes, :attachment, :string
end
end
Don't forget to rename the class and change the column details to fit your purpose. Run it.
1) Deliver attachments through Rails
The first way is to store your Carrierwave attachments not ...
The Ruby Object Model
In Ruby (almost) everything is an Object
. While this enables a lot of powerful features, this concept might be confusing for developers who have been programming in more static languages, such as Java or C#. This card should help understanding the basic concepts of Ruby's object model and how things behave.
Usage of objects in Ruby
When working with objects in Ruby, you might think of a "container" that holds metadata, variables and methods. Metadata describes stuff like the object's class or its object_id
whi...