Preventing users from uploading malicious content
When you allow file uploads in your app, a user might upload content that hurts other users.
Our primary concern here is users uploading .html or .svg files that can run JavaScript and possibly hijack another user's session.
A secondary concern is that malicious users can upload executables (like an .exe or .scr file) and use your server to distribute it. However, modern operating systems usually warn before executing files that were downloaded from t...
Compare library versions as "Gem::Version" instances, not as strings
Sometimes we have to write code that behaves differently based on the version of a specific gem or the Ruby Version itself. The version comparison can often be seen with simple string comparison like so.
# ❌ Not recommended
if Rails.version > '6.1.7.8' || RUBY_VERSION > '3.1.4'
raise Error, 'please check if the monkey patch below is still needed'
end
If you are lucky, the version comparison above works by coincidence. But chances are that you are not: For example, Rails version 6.1.10.8 would not raise an error in the code ...
Project management best practices: Technical debt summary
Maintaining larger projects makes it more difficult to balance refactoring and upgrade tasks according to its actual value. Consider to create and periodically maintain a summary, which helps you and your team in the decision which refactoring task should be taken next.
Template
Here is an template on how you might categorize your tasks:
| Technical debt | Estimated Efforts | Visible customer value| Customer value explained| Developer value|Developer value explained|
|-----------------------------|----------------|----------...
Popular mistakes when using nested forms
Here are some popular mistakes when using nested forms:
- You are using
fields_forinstead ofform.fields_for. - You forgot to use
accepts_nested_attributesin the containing model. Rails won't complain, but nothing will work. In particular,nested_form.objectwill benil. - The
:reject_ifoption lambda in youraccepts_nested_attributescall is defined incorrectly. Raise the attributes hash given to your:reject_iflambda to see if it looks like you expect. - If you are nesting forms into nested forms, each model involved ne...
How to not die with ActionView::MissingTemplate when clients request weird formats
When HTTP clients make an request they can define which response formats they can process. They do it by adding a header to the HTTP request like this:
Accept: application/json
This means the client will only understand JSON responses.
When a Rails action is done, it will try to render a template for a format that the client understand. This means when all you are HTML templates, a request that only accepts application/json will raise an error:
An ActionView::MissingTemplate occurred in pages#foo:
Missing templa...
Ruby tempfiles
With the the Ruby Tempfile class you can create temporary files. Those files only stick around as long as you have a reference to those. If no more variable points to them, the GC may finalize the object at some point and the file will be removed from the file system. In other words: tempfiles are removed automatically. If you would then try to access the tempfile using its path (which you stored previously), you would get an error because the file no longer exists.
You can proactively unlink your tempfiles to delete them earlier...
Writing a README for a project
Rails applications and ruby gems should have a README that gives the reader a quick overview of the project. Its size will vary as projects differ in complexity, but there should always be some introductory prose for a developer to read when starting on it.
Purpose
That's already the main purpose of a project README: Give a new developer a quick overview of the project. In sketching this outline, the README should notify the reader of any peculiarity he needs to know of.
Remember that in a few months, you'll be a kind of "new ...
CI Template for GitHub Actions
Usually our code lives on GitLab, therefore our documentation for CI testing is extensive in this environment. If you are tied to GitHub e.g. because your customer uses it, you may use the following GitHub Actions template for the CI integration. It includes jobs for rspec (parallelized using knapsack, unit + feature specs), rubocop, eslint, coverage and license_finder.
Note that GitHub does not allow the use of YAML anchors and aliases. You can instead use [compos...
Webpack(er): A primer
webpack is a very powerful asset bundler written in node.js to bundle (ES6) JavaScript modules, stylesheets, images, and other assets for consumption in browsers.
Webpacker is a wrapper around webpack that handles integration with Rails.
This is a short introduction.
Installation
If you haven't already, you need to install node.js and Yarn.
Then, put
gem 'webpacker', '~> 4.x' # check if 4.x is still cu...
A simpler default controller implementation
Rails has always included a scaffold script that generates a default controller implementation for you. Unfortunately that generated controller is unnecessarily verbose.
When we take over Rails projects from other teams, we often find that controllers are the unloved child, where annoying glue code has been paved over and over again, negotiating between request and model using implicit and convoluted protocols.
We prefer a different approach. We believe that among all the classes in a Rails project, controllers are some of the hardest to...
Deliver Paperclip attachments to authorized users only
When Paperclip attachments should only be downloadable for selected users, there are three ways to go.
The same applies to files in Carrierwave.
Variant: Deliver attachments through Rails
The first way is to store Paperclip attachments not in the default public/system, but in a private path like storage inside the current release. You should prefer this method when dealing with sensitive data.
...
Careful when using Time objects for generating ETags
You can use ETags to allow clients to use cached responses, if your application would send the same contents as before.
Besides what "actually" defines your response's contents, your application probably also considers "global" conditions, like which user is signed in:
class ApplicationController < ActionController::Base
etag { current_user&.id }
etag { current_user&.updated_at }
end
Under the hood, Rails generates an ETag header value like W/"f14ce3710a2a3187802cadc7e0c8ea99". In doing so, all objects from that etagge...
Migrating from Elasticsearch to Opensearch: searchkick instructions (without downtime!)
General
A general overview about why and how we migrate can be found under Migrating from Elasticsearch to Opensearch
This card deals with specifics concerning the use of searchkick.
Step 1: Make Opensearch available for Searchkick
In your Gemfile
# Search
gem 'searchkick' # needs to be > 5, to use Opensearch 2
gem 'elasticsearch'
gem 'opensearch-ruby'
in config/initializers/searchkick.rb (or wherever you have configured your Searchkick settings) add:
SEARCHKICK_CLIENT_T...
Regex: Be careful when trying to match the start and/or end of a text
Ruby has two different ways to match the start and the end of a text:
-
^(Start of line) and$(End of line) -
\A(Start of string) and\z(End of string)
Most often you want to use \A and \z.
Here is a short example in which we want to validate the content type of a file attachment. Normally we would not expect content_type_1 to be a valid content type with the used regular expression image\/(jpeg|png). But as ^ and $ will match lines, it matches both content_type_1 and content_type_2. Using \A and \z will wo...
ActiveRecord: String and text fields should always validate their length
If you have a :string or :text field, you should pair it with a model validation that restricts its length.
There are two motivations for this:
- In modern Rails, database types
:stringand:textno longer have a relevant size limit. Without a validation a malicious user can quickly exhaust the hard drive of your database server. - In legacy Rails (or database schemas migrated from legacy Rails), database types
:stringand:texthad a database-side length constraint. When the user enters a longer string, the ActiveRecord valida...
Eager-loading polymorphic associations
To avoid n+1 queries, you want to eager-load associated records if you know you need to access them later on.
The Rails docs say:
Eager loading is supported with polymorphic associations.
This is true, but has some caveats.
Example
Consider the following models:
class Image < ActiveRecord::Base; end
class Video < ActiveRecord::Base; end
class PageVersion < ActiveRecord::Base
belongs_to :primary_medium, polymorphic: true # may be Image or Video
end
class Page < ActiveRecord::Base
belongs_to ...
Beware of params with non-string values (nil, array, hash)
Recent rails security updates have shown that people make incorrect assumptions about the possible contents of the params hash.
Just don't make any! Treat it as what it is: potentially unsafe user input. For example:
/pages/edit?foo= --> params == {:foo => ""}
/pages/edit?foo --> params == {:foo => nil}
/pages/edit?foo[] --> params == {:foo => [nil]} # at least in older rails 3 and in rails 2.x
Be especially wary about stuff like
User.find_by_password_reset_token(params[:password_reset_token])
I...
The many gotchas of Ruby class variables
TLDR: Ruby class variables (@@foo) are dangerous in many ways. You should avoid them at all cost. See bottom of this card for alternatives.
Class variables are shared between a class hierarchy
When you declare a class variable, it is shared between this and all descending (inheriting) classes. This is rarely what you want.
Class variables are bound at compile-time
Like unqualified constants, class variables are bound to your current scope *whe...
Clean up application servers when deploying
Our development process makes us deploy very often. As the number of releases grows, junk clogs up the hard drive of our application servers:
- Old release code
- Old
tmpfolders with compiled view templates etc. - Precompiled assets (Javascripts, images...) that no longer exist. When using the asset pipeline, Capistrano will symlink the
public/assetsdirectory toshared/assets. This is cool since we can still serve previous assets after a new release, in the window where browser caches might still have references to old assets. But i...
What we know about PDFKit
What PDFKit is
- PDFKit converts a web page to a PDF document. It uses a Webkit engine under the hood.
- For you as a web developer this means you can keep using the technology you are familar with and don't need to learn LaTeX. All you need is a pretty print-stylesheet.
How to use it from your Rails application
- You can have PDFKit render a website by simply calling
PDFKit.new('http://google.com').to_file('google.pdf'). You can then send the...
Fixing flaky E2E tests
An end-to-end test (E2E test) is a script that remote-controls a web browser with tools like Selenium WebDriver. This card shows basic techniques for fixing a flaky E2E test suite that sometimes passes and sometimes fails.
Although many examples in this card use Ruby, Cucumber and Selenium, the techniques are applicable to all languages and testing tools.
Why tests are flaky
Your tests probably look like this:
When I click on A
And I click on B
And I click on C
Then I should see effects of C
A test like this works fine...
`simple_format` does not escape HTML tags
simple_format ignores Rails' XSS protection. Even when called with an unsafe string, HTML characters will not be escaped or stripped!
Instead simple_format calls sanitize on each of the generated paragraphs.
ActionView::Base.sanitized_allowed_tags
# => #<Set: {"small", "dfn", "sup", "sub", "pre", "blockquote", "ins", "ul", "var", "samp", "del", "h6", "h5", "h4", "h3", "h2", "h1", "span", "br", "hr", "em", "address", "img", "kbd", "tt", "a", "acrony...
RSpec: Increase readability with super_diff
When handling nested hashes the RSpec output is often hard to read. Here the gem super_diff could help.
Add super_diff to your project
- Add
super_diffto your Gemfile:
gem 'super_diff'
- Require it in your
spec_helper.rb
require 'super_diff/rspec' # For Rails applications you can replace this with 'super_diff/rspec-rails'
-
Customize colors in
spec/support/super_diff.rb
SuperDiff.configure do |config|
config.ac...
RSpec: Applying stubs only within a block
When you mocked method calls in RSpec, they are mocked until the end of a spec, or until you explicitly release them.
You can use RSpec::Mocks.with_temporary_scope to have all mocks applied inside a block to be released when the block ends.
Example:
RSpec::Mocks.with_temporary_scope do
allow(Rails).to receive(:env).and_return('production'.inquiry)
puts Rails.env # prints "production"
end
puts Rails.env # prints "test"
Note that, when overriding pre-existing mocks inside the block, they are not reverted to the previously ...