Debug SAML in development using a local keycloak server
Developing or debugging SAML functionality can be a hassle, especially when you need to go back and forth with someone external who is managing the identity provider (IDP).
But you can setup a local keycloak server to act as your IDP to play around with. This might seam intimidating, but is actually quite simple when using docker and turning off some verification steps.
1. Start a keycloak instance using docker
`mkdir -p keycloak_data && docker run --network=host -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN...
Ruby: How to connect to a host with expired SSL certificate
If you need to make an HTTPS connection to a host which uses an expired certificate, do not disable certificate verifications entirely. Doing that enables e.g. man in the middle attacks.
If you accept only a single expired and known certificate, you are much less in trouble.
Setup
All the solutions described below use a verify_callback for the request's OpenSSL::X509::Store where you can specify a lambda to adjust its verification response.
Your callback must return either true or false and OpenSSL's verification result is...
Rails: Testing file downloads with request specs
tl;dr
Prefer request specs over end-to-end tests (Capybara) to joyfully test file downloads!
Why?
Testing file downloads via Capybara is not easy and results in slow and fragile tests. We tried different approaches and the best one is just okay.
Tests for file downloads via Capybara ...
- ... are slow,
- ... are fragile (breaks CI, breaks if Selenium driver changes, ...),
- ... need workarounds for your specia...
List of handy Ruby scripts to transcode different file types (often by using GPT)
It's 2024 and we have tools like ffmpeg, imagemagick and GPT readily available. With them, it's easy to convert texts, images, audio and video clips into each other.
For the everyday use without any parameter tweaking I'm using a collection of tiny scripts in my ~/bin folder that can then be used as bash functions. And: It's faster to use the CLI than interacting with a website and cheaper to use the API than buying GPT plus.. :-)
Usage
text-to-image "parmiggiano cheese wedding cake, digital art"- `text-to-audio "Yesterday I ate ...
Virtual scrolling: A solution for scrolling wide content on desktops
I recently built a screen with a very high and wide table in the center. This posed some challenges:
- Giving the table a horizontal scroll bar is very unergonomic, since the scrollbar might be far off screen.
- Making the whole page scrollable looks bad, since I don't want the rest of the UI to scroll.
- Giving the table its own vertical scrollbar and a limited height would have solved it, but felt weird, since the table was 90% of the page.
What I ended up doing is reusing the horizontal page scrollbar (which is naturally fixed at t...
Be careful when checking scopes for blankness
Today I stumbled across a pretty harmless-looking query in our application which turned out to be pretty harmful and caused huge memory usage as well as downing our passenger workers by letting requests take up to 60 seconds. We had a method that received a scope and then checked, if the scope parameter was blank? and aborted the method execution in this case.
def foo(scope)
return if scope.blank?
# Use scope, e.g.
scope.find(...)
end
We then called this method with an all scope: foo(Media::Document::Base.all). *...
CI Template for GitHub Actions
Usually our code lives on GitLab, therefore our documentation for CI testing is extensive in this environment. If you are tied to GitHub e.g. because your customer uses it, you may use the following GitHub Actions template for the CI integration. It includes jobs for rspec (parallelized using knapsack, unit + feature specs), rubocop, eslint, coverage and license_finder.
Note that GitHub does not allow the use of YAML anchors and aliases. You can instead use [compos...
Bash: How to count and sort requests by IP from the access logs
Example
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.41 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
87.140.79.42 - - [23/Jan/2024:09:00:46 +0100] "GET /monitoring/pings/ HTTP/1.1" 200 814 "-" "Ruby"
Goal
Count and sort the number of requests for a single IP address.
Bash Command
awk '{ print $1}' test.log | sort...
Bash: How to grep logs for a pattern and expand it to the full request
Example
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [4cdad7a4-8617-4bc9-84e9-c40364eea2e4] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] test
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [6e047fb3-05df-4df7-808e-efa9fcd05f87] more
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- : [53a240c1-489e-4936-bbeb-d6f77284cf38] nope
I, [2024-01-21T06:22:17.484221 #2698200] INFO -- ...
Opening a zipped coverage report with one click
Tested on Ubunut 22.04
1. Opener script
- Create a file
~/.local/bin/coverage_zip_openerwith:
#!/bin/bash
tmp_folder="/tmp/coverage-report-opener"
if [ -z "$1" ]
then
echo "Usage: coverage_zip_opener [filename]"
exit -1
fi
if ! [[ "$1" =~ ^.*Pipeline.*Coverage.*\.zip$ || "$1" =~ ^.*merged_coverage_report.*\.zip$ ]]; then
file-roller "$1"
exit 0
fi
rm -Rf $tmp_folder
unzip -qq "$1" -d $tmp_folder
index_filename=$(find /tmp/coverage-report-opener -name "index.html" | ...
Unpoly 3.7.1, 3.7.2 and 3.7.3 released
Version 3.7.0 broke some things in complex forms. Sorry for that. Concurrent user input is hard.
3.7.1
This change fixes two regressions for form field watchers, introduced by 3.7.0:
- When a change is detected while waiting for an async callback, prevent the new callback from crashing with
Cannot destructure property { disable } of null. - When a change is detected while waiting for an async callback, the full debounce delay of that new change is honored.
...
Browser debugging tricks
A list of clever debugging tricks. TOC:
- Advanced Conditional Breakpoints
- monitor() class Calls
- Call and Debug a Function
- Pause Execution on URL Change
- Debugging Property Reads
- Use copy()
- Debugging HTML/CSS
open-next-failure: An alias to speed up test debugging
Getting an entire test suite green can be a tedious task which involves frequent switches between the CLI that is running tests back to the IDE where its cause can be fixed.
The following bash aliases helped me speed up that process:
alias show-next-failure="bundle exec rspec --next-failure"
alias open-next-failure="show-next-failure || show-next-failure --format json | jq -r '.examples[0]' | jq '\"--line \" + (.line_number|tostring) + \" \" + .file_path' | xargs echo | xargs rubymine"
There is a lot going on above but the gist...
Livereload + esbuild
Getting CSS (and JS) live reloading to work in a esbuild / Rails project is a bit of a hassle, but the following seems to work decently well.
We assume that you already use a standard "esbuild in Rails" setup, and have an esbuild watcher running that picks up your source code in app/assets and compiles to public/assets; if not change the paths below accordingly.
Basic idea
We will
- use the
guard-livereloadgem as the livereload server (which send updates to the browser), - use the
livereload-jsnpm package in the browser to con...
How to (and how not to) design REST APIs · stickfigure/blog Wiki
In my career, I have consumed hundreds of REST APIs and produced dozens. Since I often see the same mistakes repeated in API design, I thought it might be nice to write down a set of best practices. And poke fun at a couple widely-used APIs.
Much of this may be "duh", but there might be a few rules you haven't considered yet.
A very compatible default CSP for Rails projects
Every modern Rails app should have a Content Security Policy enabled.
Very compatible default
The following "default" is a minimal policy that should
- "just work" for almost all applications
- give you most of the benefits of a CSP
In your config/initializers/content_security_policy.rb, set
Rails.application.config.content_security_policy do |policy|
policy.object_src :none
policy.script_src :unsafe_eval, :strict_dynamic, :https # Browsers with support for "'strict-dynamic'" will ignore "https:"
po...
How to: Upgrade CarrierWave to 3.x
While upgrading CarrierWave from version 0.11.x to 3.x, we encountered some very nasty fails. Below are the basic changes you need to perform and some behavior you may eventually run into when upgrading your application. This aims to save you some time understanding what happens under the hood to possibly discover problems faster as digging deeply into CarrierWave code is very fun...
Whitelists and blacklists
The following focuses on extension allowlisting, but it is the exact same thing for content type allowlisting with the `content_ty...
Zeitwerk: How to collapse folders in Rails
All direct child directories of app are automatically added to the eager- and autoload paths. They do NOT create a module for namespacing. This is intuitive, since there normally is no module Model, or module Controller. If you want to add a new base directory, there's no additional config needed.
Example
app
├── controllers
├── helpers
├── inputs # No config needed
├── mailers
├── models
├── uploaders # No config needed
├── util # No config needed
└── workers # No config needed
Sometimes it's handy to group files wit...
Heads Up: Selenium 4 uses a binary to determine the chromedriver
I recently stumbled over a problem that my feature tests broke in CI because of a mismatching chromedriver version.
In this specific project we have a fixed Chromium version in a Debian 12 environment instead of Chrome. The tests however used a recent chrome version instead.
$ chromedriver --version
ChromeDriver 117.0.5938.149 (e3344ddefa12e60436fa28c81cf207c1afb4d0a9-refs/branch-heads/5938@{#1539})
$ chromium --version
Chromium 117.0.5938.149 built on Debian 12.1, running on Debian 12.1
> WARN Selenium [:selenium_manager] The chromed...
RSpec: Leverage the power of Capybara Finders and Matchers for view specs
View specs are a powerful tool to test several rendering paths by their cases instead of using a more costing feature spec. This is especially useful because they become quite convenient when used with Capybara::Node::Finders and Capybara::RSpecMatchers. This allows to wirte view unit specs as you can isolate specific part...
Using Ruby's Method objects for inspecting methods
Do you remember finding where a method is defined?
I recently that Method objects are quite useful within a debugging feast to find out the currently defined internals of methods, because they are either called within the current context or because you want to learn something about the API of the current objects.
Why is this useful?
This is especially useful since Ruby is an interpreted language an...
Split your parallel tests by execution time and keep execution logs up to date
Both knapsack and parallel_tests have the option to split groups by historic execution time. The required logs for this might be outdated since you manually have to update and push them into your repository.
The following card includes an option how you can keep them consistently up to date with no extra effort locally and/or remotely.
How to always split by execution logs
Parallel Tests
The parallel_tests gem has the option flag `--group...
Git restore vs. reset for reverting previous revisions
The git doc states on the difference of these two commands:
- git-restore[1] is about restoring files in the working tree from either the index or another commit. This command does not update your branch. The command can also be used to restore files in the index from another commit.
- git-reset[1] is about updating your branch, moving the tip in order to add or remove commits from the branch. This operation changes the commit history.
git reset can also be used to restore th...
redirect_to and redirect
There are multiple ways to redirect URLs to a different URL in Rails, and they differ in small but important nuances.
Imagine you want to redirect the following url https://www.example.com/old_location?foo=bar to https://www.example.com/new_location?foo=bar.
Variant A
You can use ActionController::Redirecting#redirect_to in a controller action
class SomeController < ActionController::Base
def old_location
redirect_to(new_location_url(params.permit(:foo)))
end
end
This will:
- It will redirect with a 302 st...