How to employ and run your tests with parallel_tests to speed up test execution

Updated . Posted . Visible to the public.

When your cucumber features grow massively over time, the test execution can take a lot of time.
One easy way to speed up your test execution is to use the parallel_tests Show archive.org snapshot gem.
It comes along with some useful rake tasks that let you setup your local test environment shortly to run your features, specs or unit-tests in parallel.

Follow these steps to get it to work.

  1. Add the parallel_tests gem to your Gemfile test sections like that:

    # ./Gemfile
    
    group :development, :test do
      gem 'parallel_tests'
    end 
    
  2. Change your database.yml file. The parallel_tests gem uses the test database for spec, features, ... Notice the <%= ENV['TEST_ENV_NUMBER'] %> appended to the database name in the test section.

    # ./config/database.yml
    
    ...
    
    test: &TEST
      adapter: mysql
      database: my_database_name_test<%= ENV['TEST_ENV_NUMBER'] %>
      encoding: utf8
      username: xxx
      password: yyy
    
    cucumber:
      <<: *TEST
    
  3. Create your test databases (as many databases as your PC has CPU cores)

    rake parallel:create
    
  4. Prepare your test databases with the db schema. Don't forget to run your migrations (on the test db) before you execute this command.

    rake parallel:prepare
    
  5. Now you are ready to rumble. You can use these commands to run your features, specs or unit tests:

    rake parallel:test          # Test::Unit
    rake parallel:spec          # RSpec
    rake parallel:features      # Cucumber
    
    rake parallel:features[2]   # example how to execute cucumber features with (only) 2 cores
    

    The great thing is that the commands are bundler aware.

Some tests showed that parallel test execution (2 CPUs instead of 1) gives you a performance gain of about 25 - 30 %.
Each doubling of the CPU cores decreased the test execution time by about 50 %. Please note that this performance gain could not be measured with all tested projects.

Using parallel_tests with Selenium

Only if you have problems running multiple selenium instances use this code snippet in your env.rb:

Capybara.server_port = 8888 + ENV['TEST_ENV_NUMBER'].to_i

Furthermore if you use the headless gem to hide your selenium window, you have to disable the automatic xvfb closing since other process might still run some selenium tests in it. Due to the reuse of the xvfb instance this step is very important to avoid lots of failing features.

at_exit do
# headless.destroy   # <-- comment out this line at least (in your env.rb)
end

Issues

  • parallel_tests breaks rspec matcher =~ for array comparision\
    Solution: Use == if possible or just compare ids for example (xxx.collect(&:id) =~ [obj1, obj2, ...].collect(&:id))
Ulrich Berkmüller
Last edit
Dominik Schöler
Keywords
performance, test, parallel, speed
License
Source code in this card is licensed under the MIT License.
Posted by Ulrich Berkmüller to makandra dev (2011-05-30 09:02)