Read more

Debug flaky tests with an Unpoly observeDelay

Niklas Hasselmeyer
October 14, 2022Software engineer at makandra GmbH

The problem

Unpoly's [up-observe], [up-autosubmit] and [up-validate] as well as their programmatic variants up.observe() and up.autosubmit() are a nightmare for integration tests.

Illustration online protection

Rails Long Term Support

Rails LTS provides security patches for old versions of Ruby on Rails (2.3, 3.2, 4.2 and 5.2)

  • Prevents you from data breaches and liability risks
  • Upgrade at your own pace
  • Works with modern Rubies
Read more Show archive.org snapshot

Tests are usually much faster than the configured up.form.config.observeDelay. Therefore, it may happen that you already entered something into the next field before unpoly updates that field with a server response, discarding your changes.

The steps I wait for active ajax requests to complete (if configured) and capybara-lockstep can catch some of these cases, but not all.
Both of these only wait until executing the next step if the request already started. If unpoly is still waiting for the observeDelay to run out, there is no XHR request and the test will continue.

The solutions

Lower the observeDelay in tests

This will always help you. Your tests type faster than a human. Your tests that correctly wait for changes to happen will get faster and your tests that don't wait will get less flaky.

Signaling asynchronous work to CapybaraLockstep

You can tell CapybaraLockstep that something on your page is still executing and that it should wait for that to finish Show archive.org snapshot .

Here is a solution for the up-observe problem:

// Lock capybara tests when changes to this form or field are observed
// The given options should be the options that are passed to up.observe or up.autosubmit
// You can define an extra `taskName` option that shows up in CapybaraLockstep's debug output
// This function is always available, but only executes if CapybaraLockstep is defined (i.e. in tests)
export function lockCapybaraForObservedChanges(formOrField, options) {
  if (window.CapybaraLockstep) {
    const taskName = options.taskName || `observed changes in ${formOrField}`
    const delay = options.delay || formOrField.getAttribute('up-delay') || up.form.config.observeDelay
    return up.on(formOrField, 'input change', () => {
      window.CapybaraLockstep.startWork(taskName)
      setTimeout(() => window.CapybaraLockstep.stopWork(taskName), delay)
    })
  }
}

if (window.CapybaraLockstep) {
  // lock capybara for all changes to fields with callbacks
  up.compiler('[up-observe]', (formOrField) => {
    return lockCapybaraForObservedChanges(formOrField, { taskName: `[up-observe]: ${formOrField}` })
  });
  up.compiler('[up-autosubmit]', (formOrField) => {
    return lockCapybaraForObservedChanges(formOrField, { taskName: `[up-autosubmit]: ${formOrField}` })
  });
  up.compiler('[up-validate]', (formOrField) => {
    return lockCapybaraForObservedChanges(formOrField, { taskName: `[up-validate]: ${formOrField}` })
  });
}

This takes care of the unobtrusive [up-observe], [up-autosubmit] and [up-validate].

For uses of up.observe or up.autosubmit, you need to call the lockCapybaraForObservedChanges function directly.
You can use it like this:

import { lockCapybaraForObservedChanges } from './path/to/the/file/above'

// ...

up.observe(element, { delay: 100 }, onChange)
lockCapybaraForObservedChanges(element, { delay: 100, taskName: 'elementChanged' })

Niklas Hasselmeyer
October 14, 2022Software engineer at makandra GmbH
Posted by Niklas Hasselmeyer to makandra dev (2022-10-14 17:41)