If you get the above error when running tests in bulk (but not individually), it's actually the fault of Webmock.
Updating Webmock to version 1.7+ fixes this.