Fixing flaky E2E tests

An end-to-end test (E2E test) is a script that remote-controls a web browser with tools like Selenium WebDriver. This card shows basic techniques for fixing a flaky E2E test suite that sometimes passes and sometimes fails.

Although many examples in this card use Ruby, Cucumber and Selenium, the techniques are applicable to all languages and testing tools.

Why tests are flaky

Your tests probably look like this:

When I click on A
And I click on B
And I click on C
Then I should see effects of C

A test like this works fine...

Async control flow in JavaScript: Promises, Microtasks, async/await

Slides for Henning's talk on Sep 21st 2017.


Understanding sync vs. async control flow

Talking to synchronous (or "blocking") API

print('script start')
html = get('/foo')
print(html)
print('script end')

Script outputs 'script start', (long delay), '<html>...</html>', 'script end'.

Talking to asynchronous (or "evented") API

print('script start')
get('foo', done: function(html) {
  print(html)
})
print('script end')

Script outputs 'script start', `'...

How to add a custom dictionary to Ruby Mine

  1. Download the dictionary from http://www.winedt.org/dict.html, e.g. http://www.winedt.org/dict/de_neu.zip
  2. unzip de_neu.zip
  3. mkdir ~/Documents/dic
  4. iconv -f UTF-16 -t UTF-8 de_neu.dic -o ~/Documents/dic/de_neu_utf8.dic
  5. In RubyMine: Go to Settings (CTRL + ALT + S) > Editor > Spelling > Dictionaries and add the folder ~/Documents/dic

RSpec: How to define classes for specs

RSpec allows defining methods inside describe/context blocks which will only exist inside them.
However, classes (or any constants, for that matter) will not be affected by this. If you define them in your specs, they will exist globally. This is because of how RSpec works (short story: instance_eval).

describe Notifier do
  class TestRecord
    # ...
  end
  
  let(:record) { TestRecord.new }
  
  it { ... }
end

# TestRecord will exist here, outside of the spec!

This will come bite you at least when you try to define a ...

Import Excel files without running into memory limitations

There are several gems that make it easy to read and process xlsx files. Parsing the entire file at once however is error-prone since each cell is transformed to a ruby object - sometimes including thousands of formatted but empty cells.

As of today, I found two promising alternatives that provide a stream-based access to spradsheet rows:

  • Roo supports multiple spreadsheet types like ODS or CSV and has a quite large contributor base
  • [Creek](https://github.com/pythonicrubyis...

Custom Ruby method Enumerable#count_by (use for quick statistics)

I frequently find myself needing a combination of group_by, count and sort for quick statistics. Here's a method on Enumerable that combines the three:

module Enumerable
  def count_by(&block)
    list = group_by(&block)
      .map { |key, items| [key, items.count] }
      .sort_by(&:last)
      
    Hash[list]
  end
end

# Returns a Hash of { key => count } pairs (see below)

Just paste that snippet into a Rails console and use #count_by now!

Usage examples

  • Number of email addresses by domain:
> User.all.co...

How to use a local gem in your Gemfile

You can use local copies of gems in your Gemfile like this:

gem 'spreewald', :path => '~/gems/spreewald'

As soon as you have bundled your project with the local copy of the gem, all code changes in the copy will be available on your project. So you can for example set a debugger or add console output in the gem and use it from your project.
If you checked out the gem with your versioning tool, you can easily reset your changes afterwards or make a pull request for the gem if you improved it.

Don't commit a Gemfile with local pat...

IRB: last return value

In the ruby shell (IRB) and rails console the return value of the previous command is saved in _ (underscore). This might come in handy if you forgot to save the value to a variable and further want to use it.

Example:

irb(main):001:0> 1 + 2
=> 3
irb(main):002:0> _
=> 3
irb(main):003:0> a = _
=> 3

Caution when using the || operator to set defaults

I often see the use of || to set a default value for a variable that might be nil, null or undefined.

x = x || 'default-value'

This pattern should be avoided in all languages.

While using || works as intended when x is null or an actual object, it also sets the default value for other falsy values, such as false. false is a non-blank value that you never want to override with a default.

To make it worse, languages like JavaScript or Perl have [many more fal...

ActiveRecord::RecordNotFound errors allow you to query the :name and :id of the model that could not be found

ActiveRecord::RecordNotFound errors provide quite meaningful error messages that can provide some insight on application details. Consider the following:

ActiveRecord::RecordNotFound: Couldn't find Organisation::Membership with 'id'=12 [WHERE "organisation_memberships"."user_id" = 1]

You should probably not simply render those error messages to the user directly. Instead you you might want to re-raise your own errors. ActiveRecord::RecordNotFound provides you with methods :model and :id where you can get information about w...

Ruby: define a class with Struct.new

This card will show you a cool way to define a class using Struct.new.
A common usecase for Structs are temporary data structures which just hold state and don't provide behaviour. In many cases you could use a simple hash as a data structure instead. However, a Struct provides you with a nice constructor, attribute accessors and complains if you try to access undefined attributes. Structs are easy to compare (by attributes). A struct gives meaning to the data.

Disclaimer

Structs are great...

Rails: How to write custom email interceptors

Nowadays it is fairly easy to intercept and modify mails globally before they are sent. All you have to do is register an interceptor class which responds to .delivering_email(message). This card will show you two common use cases.

Subject prefix:

Usually you want to prefix the subject line of emails with the current environment (except production) so you can differentiate between production mails and mails from other environments. Of course a...

Working with or without time zones in Rails applications

Rails supports time zones, but there are several pitfalls. Most importantly because Time.now and Time.current are completely different things and code from gems might use one or the other.

Especially configuring an application that cares only about one time zone is a bit tricky.

The following was tested on Rails 5.1 but should apply to Rails 4.2 as well.

Using only local time

Your life will be easier if your application does not need to support time zones. Disable them like this:

config.time_zone = 'Berlin' # Your local ...

Listing all gems on a private gem server

You can use gem list to list all gems available from a remote gem server:

gem list -r --clear-sources -s 'https://user:password@gemserver.tld/'

This is useful to debug cases where Bundler complains of a gem existing in more than one gem source.

Middleman configuration for Rails Developers

Middleman is a static page generator that brings many of the goodies that Rails developers are used to.

Out of the box, Middleman brings Haml, Sass, helpers etc. However, it can be configured to do even better. This card is a list of improvement hints for a Rails developer.

Gemfile

Remove tzinfo-data and wdm unless you're on Windows. Add these gems:

gem 'middleman-livereload'
gem 'middleman-sprockets' # Asset pipeline!

gem 'bootstrap-sass' # If you want to use Bootstrap

gem 'byebug'

gem 'capistrano'
gem 'capistrano-mid...

Quickly printing data in columns on your Ruby console

Dump this method into your Ruby console to quickly print data in columns. This is helpful for e.g. comparing attributes of a set of Rails records.

def tp(objects, *method_names)
  terminal_width = `tput cols`.to_i
  cols = objects.count + 1 # Label column
  col_width = (terminal_width / cols) - 1 # Column spacing

  Array(method_names).map do |method_name|
    cells = objects.map{ |o| o.send(method_name).inspect }
    cells.unshift(method_name)

    puts cells.map{ |cell| cell.to_s.ljust(col_width) }.join ' '
  end

  nil
end

Usag...

Howto: Require a gem that is not in Gemfile

In case you want to require a gem, that is not in the Gemfile of you bundle and therefore not in your loadpath, you need to add the path manually and require the gem afterwards.

Expected error

Requiring a gem, that is not in the Gemfile or .gemspec, will cause an LoadError exception:

require 'example_gem' => LoadError: cannot load such file -- example_gem

Adding a gem to the loadpath temporary

  1. You need to install the gem

gem install 'example_gem'

  1. Then you need to require the path where the gem was install...

Storing trees in databases

This card compares patterns to store trees in a relation database like MySQL or PostgreSQL. Implementation examples are for the ActiveRecord ORM used with Ruby on Rails, but the techniques can be implemented in any language or framework.

We will be using this example tree (from the acts_as_nested_set docs):

root
|
+-- Child 1
|   |
|   +-- Child 1.1
|   |
|   +-- Child 1.2
|
+-- ...

ruby-sass: Do not use comments between selector definitions

Sass lets you easily specify multiple selectors at once like this:

.some-block
  &.has-hover,
  &:hover
    outline: 1px solid red

This will add a red outline on either real hover or when the has-hover class is present. However, adding a comment will void the definition of that line:

.some-block
  &.has-hover, // From hoverable.js <-- DON'T
  &:hover
    outline: 1px solid red

... will simply drop the &.has-hover part in ruby-sass(deprecated). [sassc](https://rubygems.org/g...