Ruby: Following redirects with the http gem ("httprb")
When making requests using the http
gem you might want to automatically follow redirects to get the desired response. This is supported but not the default behavior.
You can do so by using the .follow
method.
This follows redirects for the following status codes: 300, 301, 302, 303, 307, 308
response = HTTP.get('https://www.example.com/redirect')
response.status # => 302
response.uri.to_s # => "https://www.example.com/redirect"
response = HTTP.follow.get('https://www.example.com/redirect')
response.status # => 200
response....
Using rack-mini-profiler (with Unpoly)
Debugging performance issues in your Rails app can be a tough challenge.
To get more detailed insights consider using the rack-mini-profiler gem.
Setup with Unpoly
Add the following gems:
group :development do
gem 'memory_profiler'
gem 'rack-mini-profiler'
gem 'stackprof'
end
Unpoly will interfere with the rack-mini-profiler widget, but configuring the following works okayish:
// rack-mini-profiler + unpoly
if (process...
Ruby: How to connect to a host with expired SSL certificate
If you need to make an HTTPS connection to a host which uses an expired certificate, do not disable certificate verifications entirely. Doing that enables e.g. man in the middle attacks.
If you accept only a single expired and known certificate, you are much less in trouble.
Setup
All the solutions described below use a verify_callback
for the request's OpenSSL::X509::Store
where you can specify a lambda to adjust its verification response.
Your callback must return either true
or false
and OpenSSL's verification result is...
List of handy Ruby scripts to transcode different file types (often by using GPT)
It's 2024 and we have tools like ffmpeg, imagemagick and GPT readily available. With them, it's easy to convert texts, images, audio and video clips into each other.
For the everyday use without any parameter tweaking I'm using a collection of tiny scripts in my ~/bin
folder that can then be used as bash functions. And: It's faster to use the CLI than interacting with a website and cheaper to use the API than buying GPT plus.. :-)
Usage
text-to-image "parmiggiano cheese wedding cake, digital art"
- `text-to-audio "Yesterday I ate ...
Rails: Testing the number of database queries
There are a few tools to combat the dreaded n+1 queries. The bullet gem notifies you of missing eager-loading, and also if there is too much eager-loading. strict_loading in Rails 6.1+ forces developers to explicitly load associations on individual records, for a single association, for an entire model, or globally for all models.
But you can also actually **write spe...
Maintaining custom application tasks in Rails
Here are some hints on best practices to maintain your tasks in larger projects.
Rake Tasks vs. Scripts
- The Rails default is using rake tasks for your application tasks. These live in
lib/tasks/*
. - In case you want to avoid rake for your tasks and just use plain ruby scripts, consider
lib/scripts/*
as folder.
Keeping tasks slim
For readability and testing it's easier to keep your tasks slim.
Example:
lib/gitlab/maintenance_tasks/user_export.rb
:
module Gitlab
module MaintenanceTasks
class UserExport
...
Disable PostgreSQL's Write-Ahead Log to speed up tests
The linked article suggests an interesting way to speed up tests of Rails + Postgres apps:
PostgreSQL allows the creation of “unlogged” tables, which do not record data in the PostgreSQL Write-Ahead Log. This can make the tables faster, but significantly increases the risk of data loss if the database crashes. As a result, this should not be used in production environments. If you would like all created tables to be unlogged in the test environment you can add the following to your...
open-next-failure: An alias to speed up test debugging
Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.
The following bash aliases helped me speed up that process:
alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"
There is a lot going on above but the gist...
Capistrano 3: Running a command on all servers
This Capistrano task runs a command on all servers.
bundle exec cap production app:run cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
Code
# lib/capistrano/tasks/app.rake
namespace :app do
# Use e.g. to grep logs on all servers:
# b cap production app:run_cmd cmd='zgrep -P "..." RAILS_ROOT/log/production.log'
#
# * Use RAILS_ROOT as a placeholder for the remote Rails root directory.
# * Append ` || test $? =1;` to grep calls in order to avoid exit code 1 (= "nothing found")
# * To be able to process ...
Gitlab: How to cancel redundant pipelines
In the Gitlab settings the flag Auto-cancel redundant pipelines is enabled by default. This auto-cancels jobs that have the interruptible
setting set to true
(defaults to false
e.g. to not cancel deploys by accident).
Consider to set the interruptible
flag for test jobs to reduce the load on your runners like in the following example .gitlab-ci.yml
:
rubocop:
interruptible: true
script:
- 'bundle exec rubocop'
rspec:
int...
Migrate searchkick from Elasticsearch client to Opensearch client without Downtime
General
A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch
This card deals with specifics concerning the use of searchkick.
Step 1: Make Opensearch available for Searchkick
In your Gemfile
# Search
gem 'searchkick' # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'
in config/initializers/searchkick.rb
(or wherever you have configured your Searchkick settings) add:
SEARCHKICK_CLIENT_T...
Livereload + esbuild
Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.
We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets
and compiles to public/assets
; if not change the paths below accordingly.
Basic idea
We will
- use the
guard-livereload
gem as the livereload server (which send updates to the browser), - use the
livereload-js
npm package in the browser to con...
Solving "TypeError (nil can't be coerced into Integer)" in the Rails console / IRB
On the Rails console, assigning an object to a variable can lead to this strange error (without stacktrace):
irb > recipient = Recipient.find(123)
Traceback (most recent call last):
TypeError (nil can't be coerced into Integer)
irb > recipient
#<Recipient ...
The error is only in the output – the assignment is working. It only occurs when using the --nomultiline
option, and thus [only with IRB 1.2.0+ and before Ruby 3](https://github.com/makandra/geordi/blob...
Debug MiniMagick calls in your Rails app
Most of our applications use CarrierWave for file uploads. CarrierWave has an integrated processing mechanism for different file versions with support for ImageMagick through CarrierWave::MiniMagick
(which requires the mini_magick
gem). In case your processing runs into an error, CarrierWave will just swallow it and rethrow an error with a very generic message like Processing failed. Maybe it is not an image?
which does not help you finding out what the actual problem is. CarrierWave probably does this for security purposes, but does n...
Spreewald, Cucumber: Selector for the nth element
The recommended additional setup of the spreewald gem, a useful set of cucumber steps, includes adding a file for defining custom selectors which can be used as prose within steps:
When I follow "Edit" within the controls section
Where the controls section
can be any arbitrary defined css selector within selectors.rb
Often it can be useful to select the nth element of a specific selector. Luckily, this can ...
How to make sure that manual deploy tasks (scheduled in Pivotal Tracker) are executed on deploy (with Capistrano)
We regularly have tasks that need to be performed around a deploy. Be it to notify operations about changed application behavior, be it to run a little oneline script after the deploy. Most database-related stuff can be handled by migrations, but every once in a while, we have tasks that are much easier to be performed manually.
Writing deploy tasks
Here is how we manage the deploy tasks themselves:
- Deploy tasks are written inside the Pivotal Tracker story description, clearly marked (e.g. with a headline "Deploy task")
- We disting...
RSpec: Efficiently rerunning failed examples during development
Note
Don't use reruns as a mean to work around flaky tests. You should always try to fix those instead of rerunning them regularly.
Setup
Configure RSpec to persist the result of your test runs to a file. This is necessary to be able to rerun examples.
Add this to your spec/spec_helper.rb
:
config.example_status_persistence_file_path = 'spec/examples.txt'
Rerun all failed examples using --only-failures
bundle exec rspec --only-failures
(or `...
Use rbenv-each to run a command for every installed Ruby version
The linked rbenv plugin rbenv-each is very helpful to keep QoL gems up to date that are not part of the Gemfile.
For example, you can bump the geordi
version for all your rubies with the following command:
rbenv each gem update geordi
Another useful example would be to bulk-update bundler
or rubygems.
Note that rbenv-each
hasn't been updated since 2018, but it is fully functiona...
Don't use log level :debug in your production environments
Catch phrase
You don't want sensitive user data in your logs.
Background
Rails per default filters sensitive data like passwords and tokens and writes [FILTERED]
to the logs. The code which is responsible for enabling that usually lives in filter_parameter_logging.rb
(Rails.application.config.filter_parameters
). Here is an example of a filtered log entry:
Unfiltered:
`User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."token" = $1 LIMIT $2 [["token", "secret-token"], ["LIMIT", 1]]`
After the filter is appl...
Split your parallel tests by execution time and keep execution logs up to date
Both knapsack
and parallel_tests
have the option to split groups by historic execution time. The required logs for this might be outdated since you manually have to update and push them into your repository.
The following card includes an option how you can keep them consistently up to date with no extra effort locally and/or remotely.
How to always split by execution logs
Parallel Tests
The parallel_tests
gem has the option flag `--group...
Lightning Talk: Coverage based Test Case Prioritization in Ruby on Rails
For my computer science bachelor's thesis I programmed and evaluated a CLI Test Case Prioritization (TCP) tool for makandra. It has been written as a Ruby Gem and was tested and evaluated against one Ruby on Rails project. This card will summarize and present the research results, the evaluation and the programmed CLI tool.
The code has been published for educational purposes on GitHub. The german bachelor's thesis has also been included for download at the end.
...
Do not pass params directly into url_for or URL helpers
Rails' url_for
is useful for generating routes from a Hash, but can lead to an open redirect vulnerability.
Your application's generated route methods with a _url
suffix are also affected because [they use url_for
unter the hood](https://github.com/rails/rails...
CarrierWave: Processing images with libvips
When you write your next CarrierWave uploader, consider processing your images with libvips instead of ImageMagick.
Reasons for libvips
There are several upsides to using libvips over ImageMagick:
- libvips is considerably faster and uses less memory.
- ImageMagick has a large attack surface that has repeatedly caused security incidents in the past (compare [ImageMagick CVEs](https://www....
RubyMine: Fixing "Rubocop returned exit code -1. See the logs for details"
When RubyMine reports Rubocop returning "exit code -1", upgrading Rubocop can be the fix:
gem install rubocop
"The logs" can be accessed with Ctrl + Shift + A > Show log in (Files). This opens your file manager at the IDEA log location; the log file is called "idea.log".
If your operating system supports it, right click into the file manager > Open in Terminal. There run tail -f idea.log
to follow the log.