An end-to-end test (E2E test) is a script that remote-controls a web browser with tools like Selenium WebDriver. This card shows basic techniques for fixing a flaky...
...and seed per process: DISPLAY=:17 bundle exec parallel_cucumber --serialize-stdout --verbose-process-command --test-options '--order random' features/ bin/cucumber --order random --profile parallel features/location/import.feature features/location/export.feature Using the parallel...
A flaky test is a test that is often green, but sometimes red. It may only fail on some PCs, or only when the entire test suite is run.
...are many causes for flaky tests. This card focuses on a specific class of feature with heavy side effects, mostly on on the UI. Features like the following can amplify...
When ending a Selenium test Capybara resets the browser state by closing the tab, clearing cookies, localStorage, etc. It may be a good idea to wait for all in-flight...
...If that JavaScript opens an error alert or spams errors to the console, your test may fail after the last step. With unlucky timing the server may receive an AJAX...
Download buttons can be difficult to test, especially with Selenium. Depending on browser, user settings and response headers, one of three things can happen: The browser shows a "Save as...
...through Selenium. The browser automatically downloads the file without prompting the user. For the test it looks like nothing has happened. The browser shows a binary document in its own...
A JavaScript error in an E2E test with Selenium will not cause your test to fail. This may cause you to miss errors in your frontend code. Using the BrowserConsole...
...helper below you can check your browser's error console from your E2E tests. The following will raise BrowserConsole::ErrorsPresent if there is an error on the browser console: BrowserConsole.assert...
...DOM elements often leave elements in the DOM after they're done. This will leak test-local DOM state to subsequent tests. For example, this test creates a element, runs...
...butler did it') }) }) To address this I like to configure a global container for test elements, which is cleared automatically after each test. By including the following helper hooks your...
The default configuration of Rails disables CSRF protection in tests. If you accidentally forget to send the CSRF token for POST requests, your tests will be green even though your...
...application is broken. You probably want to enable CSRF protection in tests that can speak JavaScript. For RSpec feature tests Add this to any file to the spec/support folder: RSpec.configure...
...Looking at the hidden browser Option 1: Run without headless You can call your tests with NO_HEADLESS=1 bundle exec cucumber to see the Chrome window for this test...
...fork TCP4:127.0.0.1:9223 (if missing, install the socat package) Start you integration test e.g. REMOTE_DEBUGGING=1 bundle exec rspec spec/system/some_spec.rb with a debugger set Open chrome://inspect/. It...
Timecop is a great gem to set the current time in tests. However, it is easy to introduce flakyness to your test suite when you forget to reset the time...
...after the test. This might be the case if: a test freezes time and a later test does not work for frozen time a later test needs the real current...
Jasmine is a great way to unit test your JavaScript components without writing an expensive end-to-end test for every small requirement. After we integrated Jasmine into a Rails...
...app we often add an E2E test that opens that Jasmine runner and expects all specs to pass. This way we see Jasmine failures in our regular test runs.
...you want to spy on a function that is imported by the code under test. This card explores various methods to achieve this. Example We are going to use the...
// client.js import { hello } from 'lib' function helloTwice() { hello() hello() } export helloTwice In our test we would like to show that helloTwice() calls hello() twice. We find out that it...
...Understand the gem Make your changes Bump the version number Update the changelog Run tests and commit Make a pull request, or release the gem Let's look at these...
...version switch, an "if Rails version < 4 do this, else do that". Gemika (see Tests below) can help you with that, but still keep these parts as short as possible...
When your Rails application server raises error, Capybara will fail your test when it clears the session after the last step. The effect is a test that passes all steps...
Validations should be covered by a model's spec. This card shows how to test an individual validation. This is preferrable to save an entire record and see whether it...
Recipe for testing any validation In general any validation test for an attribute :attribute_being_tested looks like this: Make a model instance (named record below) Run validations...
In most projects I know, Cucumber test suite speed is not an issue. Of course, running 350 features takes its time, but still each test for itself is reasonably fast...
...There is nothing you can do to fundamentally speed up such a test (of course, you should be using parallel_tests). However, in projects that go beyond clicking around in...
...of how RSpec works (short story: instance_eval). Negative example: describe Notifier do class TestRecord < ApplicationRecord # DO NOT do this! # ... end let(:record) { TestRecord.new }
end # TestRecord will exist here...
...class with the same name in another spec. Generally speaking, you don't want tests to pollute the global namespace in the first place. Below are three examples how you...
item 2 This card compares various approaches to fabricating DOM elements for testing. Constructing individual elements While you can use standard DOM functions to individually create and append...
A very visual method is to embedded a string of HTML into your test. My specs often have a function htmlFixtures() that parses the HTML and returns a reference...
...your timestamp holds milliseconds, they are discarded. While that is fine for humans, automated tests may fail because of that: If two tests run within the same second frame, your...
...application's response will have the same Last-Modified header value, even when both tests actually have different data. While test browsers usually clear all data (cookies, LocalStorage, etc.), at...
...limited to 3 e-mails per day. Further reading ActionMailer: How to send a test mail directly from the console Debugging SPF records
...so you can build on previous scripts for a similar task You can have tests (see below) Although not part of the application, your script is code and should adhere...
...ActiveRecord::RecordInvalid # Handle error end On error, log the record for manual care. Consider testing your script Not every script needs a test. However, sometimes you'll want to add...
As a default Angular CLI auto-generates test bootstrap via angular:test-bed-init via injecting it as a dynamic virtual module. Custom Bootstrap
...test environment providers Register custom Jasmine matchers and test utilities Load icons, themes, and other test-wide setup Replace CLI's auto-generated bootstrap with your own Important: Keep TestBed...
Test-Driven Development (TDD) in its most dogmatic form (red-green-refactor in micro-iterations) can be tedious. It does not have to be this way! This guide shows a...
...start writing them out and implementing them. As with the integration tests, the order of test-writing and implementation is up to you. When you're done, return to writing...
...integration of Pivotal's excellent license_finder that validates all dependencies during a regular test run. It will protect you from accidentally adding libraries with problematic licenses. Installation
LicenseFinder can exclude certain dependency groups from its report, for example development and test (add devDependencies for yarn). This only works with certain package managers (among which Bundler and...
Both knapsack and parallel_tests have the option to split groups by historic execution time. The required logs for this might be outdated since you manually have to update and...
...no extra effort locally and/or remotely. How to always split by execution logs Parallel Tests The parallel_tests gem has the option flag --group-by to change the default splitting...