If you get the above error when running tests in bulk (but not individually), it's actually the fault of Webmock.
Save money by migrating from AWS to our fully managed hosting in Germany.
Updating Webmock to version 1.7+ fixes this.