Ruby: Generating and parsing JSON, or: understanding JSON::ParserError "unexpected token"

json is part of the standard library of Ruby and deals with JSON, obviously. As you know, JSON is the string format that represents simple data structures. Ruby data structures that resemble Javascript objects can be serialized to JSON with #to_json. These can be restored from a JSON string with JSON.parse().

So what could go wrong here?

JSON.parse("a".to_json)

It will raise JSON::ParserError (784: unexpected token at '"a"'). But why?

Generating JSON vs serializing objects

J...

Carrierwave: Built-in RSpec matchers

CarrierWave comes with some RSpec matchers which will make testing more comfortable. Let's say you have an Uploader like this:

class MyUploader < CarrierWave::Uploader::Base
  include CarrierWave::MiniMagick

  # Create different versions of your uploaded files:
  version :small do
    process resize_to_fill: [100, 100]
  end
  
  version :medium do
    process resize_to_fit: [200, nil]
  end
  
  version :large do
    process resize_to_limit: [400, 400]
  end

end

Imagine you have a class Movie with an attribute poster. In ...

Geordi 5.4.0 released

5.4.0 2021-02-01

Compatible changes

  • Add geordi branch command that checks out a feature branch based on a story from Pivotal Tracker
  • Faster MySQL dumping with --single-transaction and --quick
  • Allow pivotal tracker ids in the global config file
  • Fix missing require for Fileutils in the dump load command (#145)
  • Document PARALLEL_TEST_PROCESSORS

Parsing JSON with edge cases

The linked article shows that there are unclear parts in the JSON specification and that different parsers treat them differently (which could lead to security vulnerabilities in certain cases).

I was curious what Ruby does (Ruby 2.6.6 with gem json 2.3.0, implementing RFC 7159):

Duplicate Keys

irb(main):001:0> require 'json'
=> true
irb(main):002:0> JSON.parse('{"qty": 1, "qty": -1}')
=> {"qty"=>-1}

Character Collision

irb(main):009:0> JSON.parse('{"qty": 1, "qty\ud800": -1}')
JSON::ParserError (487: incomplete sur...

Webmock < 3.12.1 cannot handle IPv6 addresses correctly

We had the issue, that a VCR spec failed, after updating CarrierWave from version 0.11.0 to 1.3.2.
In this version, CarrierWave uses the gem SsrfFilter, which retrieves the IP addresses for the given hostname and replaces the hostname in the requested url with one of them.

It works with IPv4 addresses, but not with IPv6 addresses, because WebMock cannot handle those correctly:

uri = "#{protocol}://...

Making ZSH the default shell on Ubuntu 20.04

ZSH is an alternative command line shell that includes some features like spelling correction, cd automation, better theme, and plugin support. You can replace Bash with ZSH like following:

sudo apt-get install zsh

Setting ZSH as default login shell

sudo usermod -s /usr/bin/zsh $(whoami)

Opening a new terminal window will show you a dialog where you can configure your initial ZSH config (Option 2 recommended).

Afterwards you can install the plugin manager Oh-My-ZSH and select a prop...

CarrierWave: Default Configuration and Suggested Changes

CarrierWave comes with a set of default configuration options which make sense in most cases. However, you should review these defaults and adjust for your project wherever necessary.

You will also find suggestions on what to change below.

Understanding the default configuration

Here is the current default config for version 2:

config.permissions = 0644
config.directory_permissions = 0755
config.storage_engines = {
  :f...

Mixed Content Examples

The pages […] allow you to see different types of mixed content and test how they behave in your browser. The "Secure" pages are referencing assets with HTTPS, the "Non-Secure" pages are referencing them with HTTP. Generally, you'll observe the same behavior with both Secure pages and the Secure HTTP page for a given test; the behavior will change on the Non-Secure HTTPS page.

Also see Testing HTTPS with badssl.com.

Rails: Parsing a time in a desired timezone

Sometimes you want to have a time in a given timezone independent from you Rails timezone settings / system timezone. I usually have this use case in tests.

Example

Time.parse('2020-08-09 00:00') will return different results e.g. 2020-08-09 00:00:00 +0200 depending on the Rails timezone settings / system timezone. But in this example we always want to have the given time in UTC because that's what the API returns.

it 'returns a valid API response', vcr: true do
  expect(client.get('/users/1')).to have_attributes(
    name: 'So...

Javascript: Comparing two arrays for equality

Don't Google this, you will lose all will to live. Instead use Object#isEqual() from Lodash or Underscore.js:

_.isEqual([1, 2], [2, 3]) // => false
_.isEqual([1, 2], [1, 2]) // => true

If your project already uses Unpoly you may also use up.util.isEqual() in the same way:

up.util.isEqual([1, 2], [2, 3]) // => false
up.util.isEqual([1, 2], [1, 2]) // => true

To compare two arrays for equality in a Jasmine spec assertion, see [Jasmine: Test...

How to fix: WrongScopeError when using rspec_rails with Rails 6.1

tl;dr: Upgrade the gem to at least 4.0.1

When you use rspec_rails in a version < 4 with Rails 6.1 you may encounter an error like this:

Failure/Error:
  raise WrongScopeError,
    "`#{name}` is not available from within an example (e.g. an " \
    "`it` block) or from constructs that run in the scope of an " \
    "example (e.g. `before`, `let`, etc). It is only available " \
    "on an example group (e.g. a `describe` or `context` block)."
    `name` is not available from within an example (e.g. an `it` block) or from constructs that...

Best practices for REST API design

A rough guide how to implement a REST API.


The discussion here includes some interesting points as well:

  • Timestamps: ISO8601 format ("2021-02-22T20:34:53.686Z")
  • Google API guideline: https://google.aip.dev/
  • Numbers: String vs. Number

    The JSON number type is not a double. It's just a number of arbitrary size and precision in integer/decimal/E format that can be parsed as whatever the parser finds fitting.

  • Pagination: Limit + Offset vs. Object ID / Pointer vs. System-Version...

Running old ImageMagick versions in a Docker container

If your project depends on an old version of ImageMagick that you can no longer install in your system, you can choose the run an old ImageMagick in a Docker container.

Dockerized ImageMagick commands will only work with absolute path arguments. You need to boot a corresponding docker container once before using it.

Setting up Docker

If you haven't installed Docker yet, use our guide or the [official instructions](https://docs.docker.com/get-started/...

Using ActiveRecord with threads might use more database connections than you think

Database connections are not thread-safe. That's why ActiveRecord uses a separate database connection for each thread.

For instance, the following code uses 3 database connections:

3.times do
  Thread.new do
    User.first # first database access makes a new connection
  end
end

These three connections will remain connected to the database server after the threads terminate. This only affects threads that use ActiveRecord.

You can rely on Rails' various clean-up mechanisms to release connections, as outlined below. This may...

Make your Rails console (and irb) output better readable

Pour color on your Rails console with awesome_print. Turn confusing long strings into formatted output. Have objects and classes laid out clearly whenever you need it.

Put gem 'awesome_print', :group => :development into your Gemfile. Now on the Rails console you have the command ap that will give you a colored, formatted output of whatever you pass it. See the example output of the User class below.

For customization visit the repository on Github.

![awesome_print.png](https://makan...

Some tips for upgrading Bootstrap from 3 to 4

Recently I made an upgrade from Bootstrap 3 to Bootstrap 4 in a bigger project. Here are some tips how to plan and perform such an upgrade. The effort will scale with the size of the project and its structure. If your stylesheets already follow strict rules, it may take less time to adapt them to the new version.

Preparation

There are several gems and libraries that works well with bootstrap or provide at least stylesheets/plugins to easily integrate the bootstrap theme. But very often they only work with specific version or are no long...

ActiveType #change_association: Define { autosave: true } on parent models

Consider the following models and form models:

class Parent < ApplicationRecord
  has_many :children, class_name: 'Child', foreign_key: 'parent_id'
end

class Parent::AsForm < ActiveType::Record[Parent]
  change_association :children, class_name: 'Child::AsForm', foreign_key: 'parent_id', autosave: true, inverse_of: :parent

  accepts_nested_attributes_for :children
  validates_associated :children
end
class Child < ApplicationRecord
  belongs_to :parent, inverse_of: :children
end

class Child::AsForm < ActiveType::Reco...

How to fix: Version in "./docker-compose.yml" is unsupported

When running an older version of docker-compose you might see the following error:

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
...

Shortcut for getting ids for an ActiveRecord scope

You can use .ids on an ActiveRecord 4+ scope to pluck all the ids of the relation

User.where("users.name LIKE 'Foo Bar'").ids

# same as (since Rails 3.2)
User.where("users.name LIKE 'Foo Bar'").pluck(:id)

If you are stuck on Rails 2.3 you can use Edge Rider's collect_ids:

User.where("users.name LIKE 'Foo Bar'").collect_ids(:id)

WYSIWYG with Action Text

Rails 6 includes a WYSIWYG editor, Action Text. It works out of the box quite well, but chances are that you want to add some custom functionality. This card contains some tips how to achieve this.

Setup

Basically, follow the guide in the Rails documentation. The automated script may not work with the way webpacker is configured in your project, but it should be easy to fix.

If you don't want the default c...

Rails: Postgres Partial Indexing

PostgreSQL has partial indexes. With a partial index you tell Postgres to only index rows matching a given query.

Some uses cases for a partial index:

  • You want a uniqueness index, but only under some conditions
  • Your table is very large and indexing every row becomes expensive

The linked article shows how to create a partial index with Rails.

How to fix "Command "webpack" not found"

I just ran into this deployment error after switching from the asset pipeline to webpack:

01:05 deploy:assets:precompile
      01 bundle exec rake assets:precompile
      01 Compiling...
      01 Compilation failed:
      01 yarn run v1.22.5
      01 error Command "webpack" not found.
rake stderr: Nothing written

The problem is not related to the "webpack" dependency. You probably just forgot to add a binstub to run "yarn install":

Add these lines to "bin/ya...

InfoQ: How to Design a Good API & Why it Matters

A well-written API can be a great asset to the organization that wrote it and to all that use it. Given the importance of good API design, surprisingly little has been written on the subject. In this talk (recorded at Javapolis), Java library designer Joshua Bloch teaches how to design good APIs, with many examples of what good and bad APIs look like.

Rails: How to use custom flash types in controllers

Rails supports alert and notice as default flash types. This allows you to use these keys as options in e.g. redirect_to and as a helper in views e.g. <%= notice %> (instead of flash[:notice]).

class SomeController < ApplicationRecord
  def create
    @user = User.create!
    
    redirect_to user_path(@user), notice: "#{@user} created!" 
  end
end

In case you are using Bootstrap as CSS framework you might also want to support flashes like success. This can be done with the add_flash_types method.

class Applicat...