If you get the above error when running tests in bulk (but not individually), it's actually the fault of Webmock.
Rails LTS provides security patches for old versions of Ruby on Rails (2.3, 3.2, 4.2 and 5.2)
Updating Webmock to version 1.7+ fixes this.