Debug flaky tests with an Unpoly observeDelay

Updated . Posted . Visible to the public.

The problem

Unpoly's [up-observe], [up-autosubmit] and [up-validate] as well as their programmatic variants up.observe() and up.autosubmit() are a nightmare for integration tests.

Tests are usually much faster than the configured up.form.config.observeDelay. Therefore, it may happen that you already entered something into the next field before unpoly updates that field with a server response, discarding your changes.

The steps I wait for active ajax requests to complete (if configured) and capybara-lockstep can catch some of these cases, but not all.
Both of these only wait until executing the next step if the request already started. If unpoly is still waiting for the observeDelay to run out, there is no XHR request and the test will continue.

The solutions

Lower the observeDelay in tests

This will always help you. Your tests type faster than a human. Your tests that correctly wait for changes to happen will get faster and your tests that don't wait will get less flaky.

Signaling asynchronous work to CapybaraLockstep

You can tell CapybaraLockstep that something on your page is still executing and that it should wait for that to finish Show archive.org snapshot .

Here is a solution for the up-observe problem:

// Lock capybara tests when changes to this form or field are observed
// The given options should be the options that are passed to up.observe or up.autosubmit
// You can define an extra `taskName` option that shows up in CapybaraLockstep's debug output
// This function is always available, but only executes if CapybaraLockstep is defined (i.e. in tests)
export function lockCapybaraForObservedChanges(formOrField, options) {
  if (window.CapybaraLockstep) {
    const taskName = options.taskName || `observed changes in ${formOrField}`
    const delay = options.delay || formOrField.getAttribute('up-delay') || up.form.config.observeDelay
    return up.on(formOrField, 'input change', () => {
      window.CapybaraLockstep.startWork(taskName)
      setTimeout(() => window.CapybaraLockstep.stopWork(taskName), delay)
    })
  }
}

if (window.CapybaraLockstep) {
  // lock capybara for all changes to fields with callbacks
  up.compiler('[up-observe]', (formOrField) => {
    return lockCapybaraForObservedChanges(formOrField, { taskName: `[up-observe]: ${formOrField}` })
  });
  up.compiler('[up-autosubmit]', (formOrField) => {
    return lockCapybaraForObservedChanges(formOrField, { taskName: `[up-autosubmit]: ${formOrField}` })
  });
  up.compiler('[up-validate]', (formOrField) => {
    return lockCapybaraForObservedChanges(formOrField, { taskName: `[up-validate]: ${formOrField}` })
  });
}

This takes care of the unobtrusive [up-observe], [up-autosubmit] and [up-validate].

For uses of up.observe or up.autosubmit, you need to call the lockCapybaraForObservedChanges function directly.
You can use it like this:

import { lockCapybaraForObservedChanges } from './path/to/the/file/above'

// ...

up.observe(element, { delay: 100 }, onChange)
lockCapybaraForObservedChanges(element, { delay: 100, taskName: 'elementChanged' })

Niklas Hasselmeyer
Last edit
Niklas Hasselmeyer
License
Source code in this card is licensed under the MIT License.
Posted by Niklas Hasselmeyer to makandra dev (2022-10-14 15:41)