How to recognize CVE-2019-5418
If you get requests with values for formats
like this:
{:locale=>[:de], :formats=>["../../../../../../../../../../etc/services{{"], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :haml]}
or fails like this:
Invalid query parameters: invalid %-encoding (../../../../../../../../../etc/passwd%%0000.html)
Someone tries to exploit CVE-2019-5418.
If you use the latest Rails (or latest Rails LTS) you're...
Heads up: Capybara 3's text matchers no longer squish whitespace by default
Until Capybara 2, node finders that accept a text
option were able to find nodes based on rendered text, even if it spans over multiple elements in the HTML. Imagine a page that includes this HTML:
<div class='haystack'>
Hi!
<br>
Try to match me.
</div>
Even though the text is separated by a <br>
tag in the HTML, it is matched until Capybara 2 which used to "squish" text prior to the comparison.
# Capyabara 1 or 2
page.find(...
Capybara: Testing file downloads
Download buttons can be difficult to test, especially with Selenium. Depending on browser, user settings and response headers, one of three things can happen:
- The browser shows a "Save as..." dialog. Since it is a modal dialog, we can no longer communicate with the browser through Selenium.
- The browser automatically downloads the file without prompting the user. For the test it looks like nothing has happened.
- The browser shows a binary document in its own window, like a PDF. Capybara/Selenium freaks out because there is no HTML docum...
Ruby: Using named groups in Regex
An alternative of using a multiple assignment for a Regex are named groups. Especially when your Regex becomes more complicates it is easier to understand and to process.
Note:
- In case a string does not match the pattern,
.match
will returnnil
. - With Ruby 2.4 the result of
.match
can be transformed to aHash
withnamed_captures
. This allows you to use methods likeslice
orfetch
on the result.
Example with a mult...
Function Composition in Ruby
Along with a number of other cool new features and performance improvements, Ruby 2.6 added function composition to the Proc and Method classes. Today we’ll take a look at how this allows us to use some functional programming goodness in our Ruby code.
Tod: A Gem for handling daytime without a date
Tod is a gem for working with daytimes. That's a tuple of (hour, minute second)
without a day, month or year.
Another additional gem?
Thus SQL has a time
datatype for storing time of day in the format hh:mm:ss
, neither Ruby nor Rails themselves offer an elegant way to deal with day times.
Time
and DateTime
both handle daytime values AND calendar date, using them to only store the time of day will end in inconsistent and thus confusing data, e. g. Time.new
will initialize with the current Time in your Timezone, `DateTime.n...
When reading model columns during class definition, you must handle a missing/empty database
When doing some meta-programming magic and you want to do something for all attributes of a class, you may need to access connection
or some of its methods (e.g. columns
) during class definition.
While everything will be fine while you are working on a project that is in active development, the application will fail to boot when the database is missing or has no tables. This means that Raketasks like db:create
or db:migrate
fail on a freshly cloned project.
The reason is your environment.rb
which is loaded for Raketasks and calls...
Migration from the Asset Pipeline to Webpacker
This is a short overview of things that are required to upgrade a project from the Asset Pipeline to Webpacker. Expect this upgrade to take a few days even the diff is quite small afterwards.
Preparations
1. Find all libraries that are bundled with the asset pipeline. You can check the application.js
and the application.css
for require
and import
statements. The source of a library is most often a gem or a vendor directory.
2. Find an working example for each library in the application and write it down.
3. Find out the ver...
HTML emails with inline stylesheets and webpacker
Many mail clients do not support external style sheets. Some even require all styling inline, which means you'll have to do your styling inline. For Rails applications, you can use Roadie or premailer, which lets you keep your well-structured CSS files and do the inlining for you.
Since Roadie is now in passive maintenance mode, we go with premailer:
Include premailer in your Gemfile:
gem 'premailer-ra...
During deployment: "You are trying to install in deployment mode after changing your Gemfile"
While deploying an Ruby update to an old application these days, we encountered the following misleading error:
*** [err :: some-host.makandra.de] You are trying to install in deployment mode after changing
*** [err :: some-host.makandra.de] your Gemfile. Run `bundle install` elsewhere and add the
*** [err :: some-host.makandra.de] updated Gemfile.lock to version control.
*** [err :: some-host.makandra.de]
*** [err :: some-host.makandra.de] You have deleted from the Gemfile:
*** [err :: some-host.makandra.de] *
We found out a newe...
Why you can't use timezone codes like "PST" or "BST" for Time objects
Rails' ActiveSupport::TimeWithZone
objects have both a timezone code and offset, e.g. Thu, 28 Mar 2019 16:00:00 CET +01:00
. Ruby's stdlib TZInfo
also has time zones, but with different identifiers.
Unfortunately, not all timezone codes can be used to parse strings or to move time objects into another time zone.
Some timezone codes like CET
are supported by ActiveSupport extensions like String#in_time_zone
, while many codes will actually not work:
>> '2019-03-01 12:00'.in_time_zone('PST')
ArgumentError (Invalid Timezone: PST)
...
We now have our own memoization gem "Memoized"
We forked trusty memoizer to make two changes:
- Memoized methods now preserve their arity. Previously all memoized methods had an arity of
-1
. - Memoized methods are now faster at runtime. This will only be noticable if you call a memoized methods many times in the same request.
We published our fork as a new gem named memoized.
memoized is API-compatible to memoizer, you just need to include Memoized
instead of `M...
How to install bundler for Ruby < 2.3
Bundler 2 requires at least Ruby 2.3.0 and RubyGems 2.5.0. You might get the following error when you try to install bundler
for Ruby < 2.3:
ERROR: Error installing bundler:
bundler requires Ruby version >= 2.3.0.
To fix this error upgrade your project's ruby version or install the last supported version of Bundler for Ruby < 2.3:
gem install bundler -v '~>1'
You will also see an error if your [RubyGems versi...
RubyGems can't find bundle executable although Bundler is installed
I had this error:
> gem install bundler
Successfully installed bundler-2.0.1
1 gem installed
> bundle install
Traceback (most recent call last):
2: from /home/henning/.rbenv/versions/2.5.1/bin/bundle:23:in `<main>'
1: from /home/henning/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems.rb:308:in `activate_bin_path'
/home/henning/.rbenv/versions/2.5.1/lib/ruby/2.5.0/rubygems.rb:289:in `find_spec_for_exe': can't find gem bundler (>= 0.a) with executable bundle (Gem::GemNotFoundException)
The cause was that Bundler 2 requires RubyG...
Rails asset pipeline: Using ESNext without a transpiler
If your app does not need to support IE11, you can use most ES6 features without a build step. Just deliver your plain JavaScript without transpilation through Babel or TypeScript, and modern browsers will run them natively.
Features supported by all modern browsers include:
- fat arrow functions (
() => { expr }
) -
let
/const
class
-
async
/await
- Promises
- Generators
- Symbols
- Rest arguments (
...args
) - Destructuring
You won't be able to use import
and export
, or use npm modules.
See this [ES6 compatibility mat...
Upgrading Ruby from 1.8.7 to 2.3.5
Suggested Workflow
Set the ruby version in .ruby-version
to 2.3.5, then perform these steps one by one, fixing errors as they occur:
- Update gems as listed below, and bundle
- Boot a Rails console - see below for a list of changes you will probably need
- Run Specs with
--backtrace
option - Run Cucumber features (with Geordi's
--debug
option) - When all tests are green, look through your Gemfile and remove as many version constraints as possible.
- Boot the application in different environements to spot further issues, e...
How to create a Basic Auth header value
When doing Basic Authentication, your browser will send an "Authorization" header. Its value is simply a Base64-encoded representation of "username:password" (like when you place credentials in the URL directly). Example for "user@example.com" with password "secret":
Authorization: Basic dXNlckBleGFtcGxlLmNvbTpzZWNyZXQ=
So, in Ruby, you can create such headers like so:
Base64.strict_encode64("#{username}:#{password}")
Note that when speaking to a REST API, you should be using libraries like RestClient or HTTParty which will wrap ...
Ruby: Reading and writing CSVs
In ruby you can easily read and write CSVs with the standard CSV library class.
On top of this, you can use the gem smarter_csv for reading (not writing) CSVs in a more comfortable way:
- Keep in mind, that the development of this gem is in an unknown state and the 2.0 release seems to happen never
- The API will change completely for 2.0, so you might find a bunch of unrelated documentation for 1.2
Here is an example...
How to upgrade Rails: Workflow advice
When upgrading Rails versions -- especially major versions -- you will run into a lot of unique issues, depending on the exact version, and depending on your app.
However, it is still possible to give some generic advice on how you want to tackle the update in principle.
If you are not really confident about upgrading Rails, have a look at Rails LTS.
How many update steps?
Besides the Rails upgrade itself, you might also want to upgrade your other gems and upgrade your Ruby version.
First decide in how many st...
How to: Use Ace editor in a Webpack project
The Ace editor is a great enhancement when you want users to supply some kind of code (HTML, JavaScript, Ruby, etc).
It offers syntax highlighting and some neat features like auto-indenting.
For Webpack 3+
Integrate as described in the documentation. For example load ace Editor like this:
function loadAceEditor() {
return import(/* webpackChunkName: "ace" */ 'ace-builds/src-noconflict/ace').then(() => {
return import(/* webpackChunkName: "ace" */ 'ace-builds/webpack-r...
Regular tasks for long-running projects
When projects run for many years, they require special regular maintenance to stay fresh. This kind of maintenance is usually not necessary for small or quick projects, but required to keep long-running projects from getting stale.
You should be able to fit this into a regular development block.
Quarterly
Check which libraries need updating
As time goes by, libraries outdate. Check your software components and decide if any of it needs an update. Your main components (e.g. Ruby, Rails, Unpoly) should always be reasonably up to d...
Rubymine FileType mismatch
If your Rubymine does not recognize a file type correctly although you have entered the unmistakeable file extension like material_orders_controller.rb
, this may help you:
Causing the Problem
Sometimes you create a new file and forget to enter the ending like material_orders_controller
Rubymine handles such files per default as simple txt files.
You delete this file and create a new one with correct ending: material_orders_controller.rb
. But still Rubymine treats this file as text file, no highlighting is available.
What happene...
Exception notifier: How to provide custom data to fail mails
The exception_notification gem supports to provide custom data to e.g. the fail mail within foreground or background jobs.
ExceptionNotifier.notify_exception(_ex_, :data => {:message => "was doing something wrong"})
Still this can be blocked if you have an initializer where you override the default sections
and background_sections
option. So remember to add the data
option to the desired section if required. In case you raise an exception without a data object, the fail...
Katapult 0.5.0 released
New Features
- Deployment ready for Opscomplete
- Copying view and controller templates over to target application during
basics configuration or via new commandkatapult templates
. - "Usage" section in README rewritten: Now describes two usage scenarios.
Improvements
- Generating a fixed Gemfile.lock. Run
bundle update
after code generation to
update all gems to recent versions. - Better deployment with Webpack
- Navigation only rendered if requested
- Some minor fixes